How to Decide Your Test Distribution
Part 4 of the Software Testing Course: turning the pyramid from a picture into a decision — risk-based thinking, what each layer costs and returns, and the heuristic that settles most cases.
Part 3 argued that test distribution — how many unit, integration, and end-to-end tests you have — is a design choice, not a shape to copy off a slide. This part is about actually making that choice, for a real system, without a testing consultant in the room.
Start from risk, not from a ratio
The pyramid gives you a default distribution. Risk tells you where to deviate from it. For each meaningful part of your system, ask two questions:
- How likely is this to break? Complex logic, code that changes often, integration points, anything touched by a recent refactor, and anything with a history of bugs all break more often than stable, simple code.
- How bad is it if it does? A broken checkout, a data-corruption path, or a permission bypass is a different category from a misaligned tooltip or a typo in a rarely-seen error message.
Concentrate testing effort where likelihood and impact are both high. A payments module that's under active development gets deep coverage at every layer. A settings page that hasn't changed in a year and does nothing dangerous gets a light touch, whatever the pyramid says the ratio should be. Distribution isn't one global number — it varies across the codebase, and the variation should track risk.
What each layer costs and returns
| Layer | Cost to write | Cost to run | Cost to maintain | What it proves |
|---|---|---|---|---|
| Static analysis | ~0 (configure once) | ~0 | ~0 | Whole classes of error can't occur |
| Unit | Low | Very low | Low | A specific piece of logic is correct |
| Integration | Medium | Low–medium | Medium | Units work together across a real boundary |
| End-to-end (scripted) | High | High | High | A full journey works for a user |
| Autonomous | ~0 | Medium (per run — incl. reviewing findings) | ~0 (no suite to repair) | Broad behavioural coverage, incl. unscripted flows |
The column teams consistently underweight is maintenance. A test's lifetime cost is dominated by keeping it working as the code around it changes, not by the hour it took to write. That's why a large scripted end-to-end suite so reliably turns from asset to liability: the run cost and the maintenance cost both scale with the suite's size, and neither scales down on its own.
The heuristic that settles most cases
When you're unsure which layer a check belongs in:
Write the test at the lowest layer that can actually catch the bug.
- The tax calculation is wrong? A unit test on the calculation. Fast, pinpoint, cheap to maintain.
- The calculation is right but the API serialises the amount in the wrong currency? An integration test across that boundary — a unit test on either side would pass.
- Everything computes correctly but the "Pay" button isn't wired to its handler? That needs something driving the real UI — an end-to-end or autonomous check.
Pushing a test lower than it belongs — mocking so much of the world that you're really testing your mocks — gives false confidence. Pushing it higher than it belongs — an end-to-end test for a pure function — is slow and brittle for no added assurance. The heuristic keeps each check at the layer where it's both effective and cheap.
Where exploratory and autonomous coverage fit
The pyramid only describes scripted tests. Two things sit outside it and change the calculation:
- Exploratory testing — unscripted human investigation. Not a layer with a count; a practice you budget time for, weighted toward new features and risky changes. See Part 6.
- Autonomous testing — unscripted machine exploration. It provides broad end-to-end coverage without a script per flow, and its maintenance cost is near zero because it re-derives how to use the app each run. That lets you keep the scripted end-to-end layer thin — just the exact assertions that must hold — and let autonomous runs carry the breadth. See the autonomous testing guide and Autonomous Testing vs. Scripted Tests.
Add that layer and the shape stops being a strict pyramid: a wide base of unit tests, a solid integration layer, a thin scripted E2E cap, and a broad autonomous band running alongside it.
A worked example
A team ships a billing feature. Their distribution decision, area by area:
- Static + unit: proration maths, tax rules, currency handling, invoice line-item generation — dozens of fast tests, because the logic is intricate and a mistake costs real money.
- Integration: the billing service talking to the real database and the payment provider's sandbox — a handful, targeting the wiring where a unit test on each side would both pass while the whole thing is broken.
- Scripted E2E: exactly two tests — "an expired card is declined at checkout" and "a downgrade takes effect at the next cycle, not immediately" — because those exact assertions must never regress and are easy to state precisely.
- Autonomous: the rest of the billing UI and its flows — updating a card, viewing invoice history, changing plans — run on every release, because scripting all of it by hand would cost more in maintenance than it returns.
- Exploratory: a 60-minute session on plan-change edge cases (downgrade with usage over the new limit, upgrade mid-cycle, cancel-then-resubscribe) while the feature is new.
No global ratio produced that. Each line came from asking "what's the risk here, and what's the lowest layer that catches it?"
The takeaway
Test distribution is a per-area decision driven by risk (likelihood × impact), guided by the "lowest layer that catches the bug" heuristic, and weighted by the maintenance cost each layer carries for its whole life. The pyramid is a reasonable starting default; autonomous coverage lets you keep the expensive top layer small. Part 5 moves from how many tests to design to what each one should actually check.