Detecting and Fixing Flaky Playwright Tests

A test that fails one run in fifty is invisible to a single CI pass, so the first job is to make the flakiness reproduce on demand. Once you can trigger it reliably, the fix is almost always mechanical: replace a hard waitForTimeout() with a web-first assertion that waits on the exact condition the test cares about. This page gives a numbered, repeatable procedure for catching intermittent failures with --repeat-each, reading the flaky markers CI emits, classifying what kind of race you actually have, and converting timing-based waits into deterministic ones. It sits under Flaky Test Management, part of the wider Debugging & Test Observability guide.

Root cause: a race the test never declared

A flaky test asserts on state whose arrival time it does not control. The application finishes a fetch, commits a render, and settles a layout on a schedule set by the machine, the network, and whatever else is competing for CPU on the runner — and the test encodes an assumption about how long that takes instead of what it produces. On a warm laptop the assumption holds; on a shared CI runner with eight workers fighting over four cores, it does not.

That single distinction — duration versus condition — explains the majority of intermittent failures. Everything Playwright gives you for stability, from auto-waiting locators through the retrying matchers described in Web-First Assertions, exists to move a test from the first form to the second. The rest of this page is the mechanical procedure for finding the places where a test still asserts on duration and rewriting them.

Detect flakiness deliberately

A flaky test will not reveal itself in one run, so reproduce it under load. The --repeat-each flag runs every selected test N times in a single invocation, multiplying your chances of hitting the bad timing window:

import { test, expect } from '@playwright/test';

test('add to cart updates the badge', async ({ page }) => {
  await page.goto('/cart');
  await page.getByRole('button', { name: 'Add to cart' }).click();
  // Retrying matcher: polls the badge until it reads "1" or the timeout expires.
  await expect(page.getByTestId('cart-count')).toHaveText('1');
});

Run it fifty times against the suspect spec and watch for a single red:

npx playwright test cart.spec.ts --repeat-each=50 --workers=4 --retries=0

If even one of the fifty fails, the test is flaky. Pass --retries=0 explicitly: a retry configured for CI would silently rescue the failing attempt and hide the very signal you are hunting. Running with several workers also surfaces shared-state collisions that a single worker would never expose, because parallel copies of the same test now contend for the same fixtures, records, and accounts.

If fifty passes cleanly, do not declare victory yet. Raise the count, add workers beyond your core count to starve the browser of CPU, or narrow the run with --grep to a single test so the repeats land closer together in time. Starving the runner is the cheapest way to simulate a loaded CI machine on hardware that is far too fast to reproduce the bug on its own.

Detect flakiness in CI

CI catches the flakiness you cannot reproduce locally. When retries is set in playwright.config.ts, Playwright reruns a failed test and, if a later attempt passes, marks the result flaky rather than passed or failed. The HTML reporter lists every flaky test with the attempt that failed and the attempt that recovered, which is your queue of bugs to fix. Treat a non-zero flaky count as a build smell even when the job is green — the retry did not fix anything, it only bought you a passing badge while the race stayed in the code.

Wire the JSON reporter into the same run and you can count flaky results per spec over time. A test that flakes once a month is noise; a test that flakes in one build out of five is actively costing every engineer who has to re-read its failure. Rank the queue by that rate, not by how recently the failure appeared.

Classify the flake before you fix it

Not every intermittent failure is a missing assertion. Before rewriting anything, name which of four classes you are looking at, because each has a different signal in the trace and a different fix. A timing race shows an assertion firing against a DOM that had not yet committed the update. A shared-state collision shows correct application behaviour on data some other worker changed underneath you. A layout race shows a click that landed on a node still moving under a transition, hitting whatever was underneath. A network variance flake shows a response that arrived after the assertion's own timeout, which is a budget problem rather than a logic problem.

The classification matters because the fixes are not interchangeable. Raising a timeout to paper over a shared-state collision produces a test that fails later and more confusingly. Rewriting an assertion when the real problem is a cold API on the first request of a job hides a genuine performance regression. Deciding between waiting on the DOM and waiting on the network is its own decision, covered in Waiting for Network Idle vs Element State.

Web-first assertions versus hard waits

The root of most Playwright flakiness is a fixed-duration wait. page.waitForTimeout(200) is a bet that the application settles within 200ms — a bet you lose whenever CI is slow or the network is cold. Web-first assertions (expect(locator).toHaveText(), toBeVisible(), toBeEnabled(), and the rest) retry automatically until the condition is true or the timeout expires, so they adapt to whatever speed the run happens to have. The fix for nearly every flaky test is to delete the timeout and assert on the real end state instead.

The same rule applies to reads, not just waits. Any value pulled out of the page with textContent() or innerText() is a single sample taken at one instant, with no retry behind it. If the value is not final yet, the test compares stale data and fails at random. Push the comparison into the matcher so Playwright owns the polling:

import { test, expect } from '@playwright/test';

test('order total settles after the discount applies', async ({ page }) => {
  await page.goto('/checkout');
  await page.getByRole('button', { name: 'Apply discount' }).click();

  // Anti-pattern: one sample, no retry — reads whatever is in the DOM right now.
  // const total = await page.getByTestId('total').textContent();
  // expect(total).toBe('$90.00');

  // Web-first: the matcher re-queries until the text matches or the timeout hits.
  await expect(page.getByTestId('total')).toHaveText('$90.00');

  // Assert the end state of the whole interaction, not an intermediate step.
  await expect(page.getByRole('button', { name: 'Apply discount' })).toBeDisabled();
});

Step-by-step fix

1. Reproduce the flake locally. Run the suspect spec with npx playwright test <file> --repeat-each=50 --workers=4 --retries=0. If nothing fails, raise the repeat count or add workers until at least one run goes red, confirming the test is genuinely flaky and not a one-off infrastructure blip.
2. Capture a trace of the failure. Run with --trace on so the failed attempt records a full action timeline, then open it with npx playwright show-trace to see the exact action that flipped and whether the application or the assertion ran too early.
3. Classify the failure. Decide from the trace whether you are looking at a timing race, a shared-state collision, a layout race, or plain network variance. The class determines the fix, and applying the wrong one produces a test that regresses within days.
4. Find the hard wait or premature read. Search the spec for waitForTimeout, sleep, and any value read with textContent() or innerText() immediately after an action. These single-sample reads run before async state settles and are the usual culprits.
5. Replace it with a web-first assertion. Swap the timed wait for await expect(locator).toHaveText(...), toBeVisible(), or toHaveCount(). The assertion polls until the expectation holds, removing the race without guessing a duration.
6. Isolate any shared state. If the flake only appears with multiple workers, the test depends on data another test mutates. Provision unique data per test through a fixture, as described in Setting Up Global Fixtures for Parallel Tests, and never rely on a clean global database.
7. Re-run the repeat loop to confirm. Execute --repeat-each=50 again. A fix is proven only when the full batch passes; if a failure remains, return to the trace, because a second cause is still present.

Track every flake through the same states so nothing is silently abandoned half-fixed. A test only leaves the queue after a clean repeat loop, and if it reappears in a later build it re-enters at the top rather than being retried away.

If the fix cannot land this week — a third-party dependency, an unfixed application bug — move the test into quarantine rather than leaving it to erode trust in every red build. Quarantine is a debt marker with an owner, not a resting place.

Troubleshooting variants

It fails on CI and never locally

The runner is slower, colder, and more contended than your machine, and that difference is the whole bug. Reproduce the conditions rather than the test: run headless with more workers than cores, throttle the app's API with a route handler that delays responses, and start from an empty browser cache so the first navigation pays full cost. If it still refuses to fail locally, work from the CI artifacts instead — enable trace, video, and screenshots as described in Capturing Screenshots and Video on Failure, then download the failing shard's trace. Under sharding, confirm which shard produced the failure, since a flake that only appears on one shard usually means test ordering, not timing.

The repeat loop passes but the suite still goes flaky

Repeating one spec in isolation removes the very interference that causes the failure. The flake lives in the interaction between tests: a leftover cookie, a record another spec deleted, a rate limit shared across workers. Run the whole suite with --repeat-each=3 and full parallelism instead of hammering one file, and check whether the failure follows the test or follows the worker. If a specific worker index always fails, you have per-worker state; if the failure follows the test regardless of worker, you have an ordering dependency to break with fixture-scoped setup.

The flake moved to a different test after the fix

That is usually correct behaviour surfacing a second race that the first one masked. When the original test waited 200ms, it accidentally gave the application time to settle for everything that ran after it; removing the wait exposed the next assumption downstream. Treat the new failure as a fresh entry in the queue and run the same procedure. If the moved failure involves a React or similar component tree re-rendering mid-assertion, the patterns in Waiting Strategies for Dynamic React Components apply directly.

Verification

Confirm the fix three ways. First, the --repeat-each=50 --workers=4 loop passes with zero failures across the whole batch, run with retries disabled so nothing is rescued. Second, the HTML report shows a flaky count of zero for that spec across several CI runs, not just one green build — a single pass proves nothing about a one-in-thirty race. Third, open the trace from a passing run in the Playwright Trace Viewer and confirm the assertion now resolves after the application reaches its end state rather than racing it; the action timeline should show the poll starting before the update and completing just after it.

Watch the wall-clock time as well. A fix that swaps a fixed wait for a retrying matcher usually makes the test faster, because the assertion resolves the moment the condition holds instead of always paying the full 200ms. If the runtime went up sharply, the assertion is probably polling until near the timeout on every run, which means the condition you asserted is not the one the application actually reaches. Stepping through the run in Playwright Inspector and UI mode will show which matcher is burning the time. To lock the gains in, set the retry and timeout values described in Configuring Retries and Timeouts for Stable CI.

Frequently Asked Questions