Why End-to-End Tests Get Flaky (and How to Fix It)

A flaky test — one that passes and fails without the code changing — is worse than no test. It costs time on every false alarm, and it trains the team to rerun until green. Once that reflex sets in, the whole suite stops being trusted, and real failures start slipping through alongside the noise. End-to-end tests are the flakiest kind, and the causes are well understood enough that most flakiness is preventable.

What "flaky" means and why it matters

A flaky test produces different results on the same code under the same conditions. Run it ten times, it passes seven and fails three, and nothing changed.

The individual failure isn't the real cost — it's the erosion of trust. A suite people don't believe has negative value: it still costs time to run and maintain, but it no longer stops bugs, because a red build gets a shrug and a re-run instead of an investigation. Teams routinely spend more on managing a flaky suite than they'd spend with no E2E tests at all.

Almost all E2E flakiness traces to one of four root causes.

Cause 1: timing

The test acts before the application is ready. It clicks a button whose handler hasn't attached yet, asserts on a list that's still loading, or navigates before the previous action finished. Sometimes the app is fast enough that it works; sometimes it isn't.

Why it happens: the test assumes the app moves at a fixed speed. It doesn't — it depends on network latency, backend load, the size of the data, and what else the machine is doing.

Fix: wait on application state, never on a fixed duration. Wait for the specific element to be present and interactable. Wait for the network request that populates the list to resolve. Wait for a loading spinner to disappear. Modern frameworks have auto-waiting for exactly this; use it, and never write sleep(2000) — it's slow when the app is fast and still flaky when the app is slow.

Cause 2: shared state and test order

One test creates or modifies data that changes how a later test behaves. The suite passes when tests run in the order they were written and fails when they run in parallel or shuffled — or fails only on the second run of the day, because the first run left the database dirty.

Why it happens: tests share a database, a user account, or a tenant, and don't clean up after themselves.

Fix: every test sets up its own starting state and doesn't depend on anything another test did. Use isolated data per run — a fresh account, a unique tenant, seeded fixtures created in setup and removed in teardown. A good litmus test: shuffle the run order. If the suite can't pass in a random order, its tests are coupled, and coupled tests are flaky tests waiting to happen.

Cause 3: brittle selectors

The test locates elements by a fragile path — a deep CSS chain like div.container > div:nth-child(3) > button, an auto-generated class name, or an index into a list. A harmless refactor moves the element and the test can't find it, even though nothing a user would notice changed.

Why it happens: the selector encodes the DOM structure, which is an implementation detail that changes freely.

Fix: identify elements by what they are to a user — their role and accessible name (getByRole('button', { name: 'Save' })) — or by a stable, intentional data-testid that the team commits to keeping. Never by styling hooks, structural position, or generated classes.

Cause 4: third parties

The test depends on an external service — a payment sandbox, an email provider, an OAuth endpoint, an analytics beacon — that is occasionally slow, rate-limited, or down. Your code is fine; the dependency blinked.

Fix: stub third parties at the boundary for the majority of tests, so they exercise your integration code against a fast, deterministic fake. Keep a small number of tests that hit the real integration to catch contract changes, isolate them, run them less often, and give known-noisy calls bounded retries.

A quick diagnostic

When a test flakes, before you add a retry, ask which cause it is:

• Fails at a click or assertion, more often under load → timing.
• Fails only in parallel, or only on a second run → shared state.
• Fails right after a UI refactor with a "not found" error → selectors.
• Fails at a step that talks to an external system → third parties.

Retrying a flaky test hides all four. It should be a last resort for genuinely non-deterministic behaviour, not the default response.

Why autonomous exploration sidesteps most of this

Three of the four causes — timing, brittle selectors, and much of the state coupling — come from replaying a fixed script written against a snapshot of the app. An autonomous agent like Manta doesn't replay a script. It works out how to reach each page at run time, waits on the app reaching a stable rendered state before acting, and identifies elements by their role and purpose rather than a stored selector path. There's no fixed timeline to race and no brittle locator to break, so those classes of flake don't occur.

Genuine non-determinism in the app itself — a race condition, an occasionally-failing request, an inconsistent response — still shows up as a finding, which is correct: that's a real bug, not test flakiness.

Autonomous Testing vs. Scripted Tests covers the difference in full, and the end-to-end testing guide puts flakiness in the context of overall E2E strategy. If you're choosing tools, Manta vs Cypress gets into the flake trade-off specifically.

Bottom line

Flaky E2E tests almost always come from timing, shared state, brittle selectors, or third parties — each with a concrete, known fix. Diagnose the cause before reaching for a retry. And if you're fighting the first three constantly across a large suite, that's a signal the scripted approach is costing more than it returns for that coverage — see continuous testing for keeping the feedback loop trustworthy.

---

Tired of a suite nobody trusts? Point Manta at your app — nothing to script, so nothing to go flaky.