The Bottleneck Was Never Where We Expected It

Every growing product eventually asks the same question: will this hold up when more people show up? The instinct is to answer it with intuition — add servers, raise limits, hope for the best. Scalability testing exists to replace that instinct with evidence.

But there's a version of this discipline that gets flattened into "run a load test and see what breaks." That's only half of it. The harder half is knowing what load to test for — and knowing that your own application isn't always the thing that breaks first.

Here are four engagements that taught us that the hard way.

The launch that had one shot

In 2021, Circles partnered with KDDI — Japan's largest telecom operator — to launch povo 2.0, a fully digital, app-driven mobile brand. A national telco launch doesn't get a soft opening. Performance validation covered onboarding, payment, provisioning, and a microservices architecture running on RabbitMQ-based async workloads. povo 2.0 went on to scale from zero to over one million subscribers in its first year, about 70% of them new to the operator, while holding performance stability through that growth.

During pre-launch load testing, we found RabbitMQ consumers with incorrect prefetch configuration — reserving too many messages per consumer, which left some workers overloaded while others sat idle, driving up queue latency and onboarding delays under stress. No amount of architecture review would have caught this on paper. "Async messaging is a risk area" is a reasonable static observation. The actual misconfiguration only revealed itself once real concurrent traffic hit the system.

The launch that deliberately stayed small

Three years earlier, we worked with the same company on the opposite kind of problem. In 2018, Circles Life — a Singapore-based digital telco with about 250,000 subscribers at home — was expanding into Australia, targeting a modest ~50,000 subscribers on an entirely different cloud provider (GCP instead of AWS).

The goal wasn't "how much can this handle." It was "what's the minimum we actually need." We mapped Singapore's real traffic distribution patterns and scaled them down to a realistic Australian expectation, rather than provisioning for a hoped-for future with no data behind it. That restraint mattered: Circles Life sold its Australian business and exited the market in 2025. The infrastructure sized to the real target in 2018 meant that exit didn't leave behind years of overpaid capacity for a market that, in the end, didn't scale the way anyone might have hoped. Scalability testing here wasn't insurance against failure. It was capital discipline.

The bottleneck that wasn't in the app at all

In 2017, we tested a two-sided marketplace app for home services — plumbers, electricians — where servicers needed live maps and routing to reach jobs. During static analysis of third-party dependencies, we found the Google Maps API integration sitting on a free tier capped at 200 requests a day. That limit would be gone almost instantly under real usage — and no load test against the app's own infrastructure would have surfaced it, because the app itself was never the bottleneck. The fix wasn't code. It was a contract.

The failure nobody complains about

A quieter version of the same problem: a transactional email service, sending registration confirmations, provisioned with a fixed daily sending cap. Fine under steady-state signups. But the moment a marketing team ran an aggressive promo campaign and registrations spiked, that cap became the actual ceiling on how many new customers could complete sign-up — not the app's servers, not the database.

This failure mode is worse than a typical bottleneck because it's silent. Once the quota's hit, verification emails just stop going out. There's no error page, no support ticket. Most affected users don't complain — they just never get their confirmation email and quietly walk away. A campaign the business paid to run ends up losing most of the customers it was supposed to acquire, right at the moment it mattered most, and plenty of them never try again.

The pattern underneath all four

None of these risks were obvious from a product spec. They surfaced only because someone deliberately went looking — either by modeling realistic load and generating it, or by reading a dependency's fine print and comparing it against expected demand. That's really the two halves of scalability testing: static analysis, which reads architecture and contracts before anything runs, and dynamic testing, which validates real behavior under real concurrency. Neither replaces the other. Static analysis found the Maps API cap and the email quota — pure comparison, no traffic required. Dynamic testing found the RabbitMQ misconfiguration — something concurrency alone reveals.

But both are only as good as the workload behind them

This is the part that's easy to skip and expensive to skip badly. A perfectly executed load test against the wrong workload produces a confident, useless answer.

A real workload model has to get five things right:

Start from real data, not expectation

For Circles Life Australia, the workload wasn't invented from a market-sizing slide — it came from mapping Singapore's actual traffic patterns down to a realistic Australian scale. Evidence beats forecast whenever both are available.

Understand the intra-day shape, not the daily average

Twenty visits over ten hours isn't two an hour. Usage clusters — lunch, evenings, commute windows — and a system built for the average will fail exactly at the peak. This is why the email-quota failure hit so hard: a promo campaign is an engineered spike on top of natural daily peaks, and an average-based workload model will badly understate what that spike actually demands.

Understand seasonality, which needs a year of data

Daily shape needs a day of history. Seasonal shape needs a year. New products rarely have that yet — which means falling back on proxies (comparable markets, industry benchmarks) and being honest that it's a proxy, not measured fact.

Model journeys, not isolated hits

Login → search ×10 → cart ×2 → purchase ×1 — each step touches different systems at different cost, and the ratio matters as much as the volume.

Weight by cost, not just count

Running a report builder is not the same as opening a page. Treating every request as equivalent hides where the real strain lands.

Segment by role

A servicer viewing a live map behaves nothing like a customer booking a job. Flattening every user into one generic profile flattens the workload's real shape along with it.

A workload profile, done properly, isn't a single number. It's a model — grounded in real data, shaped over time, built from behavior, weighted by true cost, split by role. Get that wrong, and everything built on top of it — the static review, the load test, the tooling — is measuring the wrong thing with great precision.

Where the real complexity hides

Load generation tools are mature and well documented. That's not where this gets hard. The hard part is everything tooling doesn't automate: environment parity with production, the sheer cost of generating realistic load at real scale, tracing a bottleneck through a distributed system, and — this one especially — actually reading a vendor's rate limits and comparing them against your workload estimate. No tool does that last one for you.

Which is really the whole point. Tools generate load. Only a properly built workload model tells you whether that load means anything at all.

If you are planning a launch, an expansion, or a campaign that will change the shape of demand, our scalability testing work covers both halves: reading the architecture and contracts before anything runs, then validating real behavior under a workload that matches how your users actually show up.