---
title: "Erratic (Flaky) Test"
type: "test-smell"
slug: "erratic-test"
url: "http://localhost:3000/en/test-smells/erratic-test.md"
category: "Erratic Smells"
description: "An erratic (flaky) test passes and fails intermittently on the same code because its outcome depends on timing, ordering, shared state, or other nondeterministic factors rather than the behavior under test."
---
# Erratic (Flaky) Test

> An erratic (flaky) test passes and fails intermittently on the same code because its outcome depends on timing, ordering, shared state, or other nondeterministic factors rather than the behavior under test.

## Signs and Symptoms

A test that is **green on one run and red on the next without any code change** is erratic. You recognize it by the human behavior around it as much as the code: developers re-run CI "to make it pass," add `@Flaky`/`retry(3)`, or quarantine the test instead of fixing it.

Common tells in the code and the failure pattern:

* **Timing/async:** `sleep`/`setTimeout` "waits," un-awaited promises, or assertions that race the system under test. Failures correlate with machine speed or CI load.
* **Order dependency:** the test passes in isolation but fails in the full suite, or vice versa. Shuffling test order changes the result.
* **Shared mutable state:** static collections, a reused DB row, a singleton, or a fixture mutated by an earlier test.
* **Nondeterministic inputs:** `Date.now()`/`new Date()`, `Math.random()`, locale/timezone, hash-map iteration order, or auto-generated IDs.
* **External resources:** real network, filesystem, or clock. Failures look like timeouts or "connection refused," not assertion mismatches.
* **Conditional assertions:** `expect` hidden inside `if`/`catch`/callbacks, so the test silently passes when the branch never runs.

```js
// Smell: a hardcoded "wait," a shared array, and a real clock
const created = [];                       // shared across tests → interacting tests

test('shows a fresh receipt', async () => {
  render(<Checkout />);
  fireEvent.click(screen.getByText('Pay'));
  await new Promise(r => setTimeout(r, 300));   // hope the request finished
  created.push('order-1');                       // leaks into later tests
  expect(screen.getByRole('status'))
    .toHaveTextContent(`Paid ${new Date().toISOString()}`); // changes every run
});

```

This maps directly to Meszaros's _Erratic Test_ sub-smells: **Interacting Tests / Test Run War** (shared state), **Lonely Test** (only runs after another), **Resource Optimism** (assumes an external resource is there), **Resource Leakage** (doesn't clean up), and **Nondeterministic / Unrepeatable Test** (time, randomness, async).

## Reasons for the Problem

**Why it happens**

* **Implicit timing.** Async code is "synchronized" with fixed `sleep`s or by not awaiting at all. The delay is a guess: long enough today, too short under load tomorrow.
* **Hidden coupling.** Tests share a database, a singleton, module-level variables, or files on disk. One test's side effects become another's preconditions (Meszaros's _Interacting Tests_ / _Test Run War_), so results depend on order and on what ran in parallel.
* **Nondeterministic inputs leak in.** Real wall-clock time, `Math.random()`, locale/timezone, and unordered collections vary between runs and machines.
* **Optimistic dependence on the environment.** Tests hit a live network/service or assume a file exists (_Resource Optimism_) and never release what they acquire (_Resource Leakage_).
* **Assertions that can be skipped.** Putting `expect` in a conditional or a `.catch` means the check may never execute, so the test "passes" by accident.

**Why it hurts**

* **False confidence and masked defects.** A flaky test can fail for reasons unrelated to production, _and_ a real regression can hide behind a failure everyone assumes is "just flake." You can no longer tell signal from noise.
* **Erosion of trust.** Once a suite is known to be flaky, developers ignore red builds and reflexively retry, which trains the team to disregard the test suite entirely.
* **Wasted time and broken pipelines.** Retries, reruns, and "is it me or the test?" investigations slow everyone down and block CI/CD on non-issues.
* **Poor maintainability.** Erratic tests are hard to debug because the failure isn't reproducible; they tend to get disabled rather than fixed, quietly reducing real coverage.

## Treatment

Treat flakiness as a defect in the test, not a quirk to retry around. Reruns are for **detecting** flakiness, never for **hiding** it. Quarantine the test if it blocks the pipeline, then root-cause it.

**1\. Replace sleeps with condition-based waiting.** Poll for the state you actually care about instead of guessing a duration.

```js
// before — racy fixed delay
fireEvent.click(screen.getByText('Pay'));
await new Promise(r => setTimeout(r, 300));
expect(screen.getByRole('status')).toBeInTheDocument();

// after — wait for the condition, then assert
fireEvent.click(screen.getByText('Pay'));
expect(await screen.findByRole('status')).toBeInTheDocument();

```

**2\. Make inputs deterministic.** Inject the clock or use fake timers; seed or stub randomness; pin locale/timezone.

```js
// before: depends on the real date
expect(label).toBe(`Paid ${new Date().toISOString()}`);

// after: freeze time (vitest/jest)
vi.useFakeTimers();
vi.setSystemTime(new Date('2026-06-15T00:00:00Z'));

```

**3\. Isolate every test.** Give each test fresh fixtures, reset shared state (DB, singletons, module globals) in `beforeEach`/`afterEach`, and avoid module-level mutable variables. Run the suite in **randomized order** (e.g. Jest's `--seed`/randomize, JUnit `MethodOrderer.Random`) to surface ordering dependencies early.

**4\. Stub external resources.** Mock network, filesystem, and system calls so the test never depends on a service being up (cures _Resource Optimism_); always release/clean up acquired resources (cures _Resource Leakage_).

**5\. Await all async work and assert unconditionally.** Return/await promises and async test utilities; move side effects out of `waitFor` callbacks so they run once; never bury `expect` inside `if`/`catch`.

**6\. Verify the fix.** Run the now-deterministic test many times (and under load / shuffled order) to confirm stability before taking it out of quarantine.

## Detected by

- **sonar** `java:S5973` — Tests should be stable (https://rules.sonarsource.com/java/RSPEC-5973/)
- **sonar** `java:S2925` — "Thread.sleep" should not be used in tests (https://rules.sonarsource.com/java/RSPEC-2925/)
- **eslint-jest** `no-conditional-expect` — no-conditional-expect (https://github.com/jest-community/eslint-plugin-jest/blob/main/docs/rules/no-conditional-expect.md)
- **eslint-vitest** `no-conditional-expect` — no-conditional-expect (https://github.com/veritem/eslint-plugin-vitest/blob/main/docs/rules/no-conditional-expect.md)
- **eslint-testing-library** `await-async-queries` — await-async-queries (https://github.com/testing-library/eslint-plugin-testing-library/blob/main/docs/rules/await-async-queries.md)
- **eslint-testing-library** `await-async-utils` — await-async-utils (https://github.com/testing-library/eslint-plugin-testing-library/blob/main/docs/rules/await-async-utils.md)
- **eslint-testing-library** `no-wait-for-side-effects` — no-wait-for-side-effects (https://github.com/testing-library/eslint-plugin-testing-library/blob/main/docs/rules/no-wait-for-side-effects.md)