Test Run War.
A Test Run War is when tests pass for one person but fail randomly the moment several people or CI jobs run the suite at the same time, because the tests collide on a shared, persistent fixture.
##Signs and Symptoms
You recognize a Test Run War by who and when, not by the test body itself:
- The suite is green on your machine and in solo CI runs, but goes red "for no reason" when a teammate runs it concurrently, or when two CI jobs/pipelines hit the same environment.
- Failures are transient and non-reproducible — re-running the same test usually makes it pass.
- Failures cluster around a shared, mutable, persistent resource: one shared test database, a shared S3 bucket/queue/cache, or a fixed account/tenant.
- The error shapes are telltale: duplicate-key / unique-constraint violations,
expected 1 row but found 2, orrecord not found(someone else's run deleted it).
// Both tests assume they exclusively own a SHARED test database.
test('creates a user', async () => {
await db.users.insert({ id: 42, email: 'alice@example.com' }); // hardcoded key
expect(await db.users.findById(42)).toMatchObject({ email: 'alice@example.com' });
});
test('counts active users', async () => {
expect(await db.users.countActive()).toBe(1); // assumes nobody else added rows
});
When two runners hit the same database at once, the insert collides on id: 42 (duplicate key) and the global count is no longer 1. Each test passes alone; together-but-concurrent, they fight.
##Reasons for the Problem
Why it happens. The tests lean on a globally Shared Fixture — usually a single persistent resource (one shared test DB, a shared bucket, a fixed user) instead of a Fresh Fixture built per test. Hardcoded identifiers and assertions on global state quietly assume the test owns the resource. As long as exactly one run touches it at a time, the illusion holds. Add concurrency — a second developer, parallel CI jobs, or parallel test workers — and the runs interleave on the same rows and keys.
Why it hurts. Meszaros classifies this as a form of Erratic Test: it is essentially an Interacting Tests problem where the interaction is between concurrent test runs rather than between tests in one run.
- Reliability: failures are nondeterministic and depend on who else is running, so they cannot be reproduced on demand.
- False confidence & lost trust: the team learns to "just re-run it," and starts shrugging off red builds — including the real regressions hiding among the noise.
- Maintainability / debuggability: the cause is invisible in the failing test; it lives in another run, and interleaving makes root-causing brutal.
- Scalability: it blocks the two things teams most want — parallelizing the suite and growing the number of people who can run it at once.
##Treatment
Eliminate the shared, contended state. Concrete steps, roughly in order of preference:
- Prefer a Fresh Fixture. Each test creates exactly the data it needs and tears it down — or runs inside a transaction that is rolled back at teardown. Don't depend on pre-existing shared rows.
- Isolate per runner (Database Sandbox / partitioning). Give each developer and each CI job/worker its own database, schema, or namespace (schema-per-worker, an ephemeral container DB, or a branch database). Concurrent runs then simply can't see each other.
- Use Distinct Generated Values for keys. Replace hardcoded ids/emails with unique generated values (UUID, sequence, or
timestamp + workerId) so concurrent runs create distinct objects instead of colliding. - Don't assert on global counts. Scope assertions to the data this test created (filter by its unique key/tenant) rather than
count == 1. - Make external resources per-test/per-worker. Unique temp directories, unique queue/topic names, an in-memory or testcontainer DB per process.
// BEFORE: hardcoded key + global assertion against a shared DB
await db.users.insert({ id: 42, email: 'alice@example.com' });
expect(await db.users.countActive()).toBe(1);
// AFTER: unique key + scoped assertion (Fresh Fixture + Distinct Generated Values)
const id = crypto.randomUUID();
const email = `alice+${id}@example.com`;
await db.users.insert({ id, email });
expect(await db.users.findById(id)).toMatchObject({ email });
// ...and point each worker at its own schema/sandbox database
Note: there is no linter for this — it surfaces only at runtime under concurrency. Make it reproducible by deliberately running the suite twice in parallel against the same environment (or bumping test-runner workers) in CI; if that turns the suite red, you have a Test Run War to fix.