ConstructiCat Logo
CodeBust.
Browse section ▾

Mystery Guest.

A test whose inputs or expected results live in an external resource — a file, database seed, or shared fixture — so you can't understand or trust the test by reading it alone.

##Signs and Symptoms

You cannot understand a test by reading it: the data that drives it (and justifies its assertions) lives somewhere off-screen — a CSV/JSON file, a DB seed script, a shared fixture module, or a framework data-fixture annotation. The test references an opaque resource and then asserts on "magic" values whose meaning is hidden in that resource.

Tell-tale signs:

  • The body calls something like readFileSync('fixtures/subscribers.csv'), loadSeed('invoices.sql'), getResource(...), or a global beforeAll that seeds a shared database.
  • Expected values look arbitrary (toHaveLength(3), a specific id/email) and you must open another file to learn why.
  • A shared/"general" fixture is reused across many tests, so the inputs relevant to this test are buried among data it doesn't care about.
  • Tests break when a teammate edits a shared fixture for an unrelated test.
test('returns active subscribers', async () => {
  // Mystery Guest: what rows are in this file? why is 3 correct?
  const subscribers = await loadSubscribersFromCsv('./fixtures/subscribers.csv');
  const result = filterActive(subscribers);
  expect(result).toHaveLength(3); // the justification lives outside the test
});

The original example from Meszaros/the Test Smells catalog has the same shape: loadAirportsAndFlightsFromFile("test-flights.csv") followed by assertEquals(1, flightsAtOrigin.size()) — the "1" only makes sense if you read test-flights.csv.

##Reasons for the Problem

Why it happens

  • DRY taken too far. Test data is extracted into a shared file or "general fixture" to avoid duplication, trading readability for reuse.
  • Convenience with real-world data. Dumping a production CSV/JSON or a seed.sql feels easier than constructing objects inline.
  • Legacy/integration setup. Suites that boot a shared database or rely on framework data-fixture annotations (@magentoDataFixture …) inherit hidden state by default.

Why it hurts

  • Readability / cause-and-effect. The link between input and expected output is severed. A reader can't see why the assertion is correct without leaving the test, which defeats a test's role as executable documentation.
  • False confidence. You don't actually know what the test exercises; the file may contain more (or less) than you assume, so a green run proves less than it appears to.
  • Reliability / determinism. The external resource can be missing, renamed, reformatted, or differ by environment, OS, encoding, or locale — failures then reflect the fixture's state, not a real defect (flaky tests).
  • Maintainability / coupling. When several tests share one resource, anyone editing it for one test can silently break the others, and nobody knows which fields each test depends on.

##Treatment

Make the relevant input visible inside the test, next to the assertion that depends on it (a Fresh Fixture / inline setup). The goal is that the expected value becomes self-evident.

Concrete steps:

  1. Inline the data that matters. Construct the few objects/rows the test cares about in the test body, so the assertion's expected value is obviously correct.
  2. If you genuinely need a file, build it in the test. Use a helper that takes only the salient parameters and writes to a temp path, then clean up — so the meaningful values appear in the test, not in a checked-in blob.
  3. Replace shared/"general" fixtures with per-test fixtures, or expose intention-revealing creators/finders (createProductWithName('Simple Product'), getRecentlyAddedProduct()) so the attributes under test are explicit while irrelevant setup stays hidden behind a well-named builder — not behind an opaque resource.

Before → after:

// Before — Mystery Guest: data and "2" are hidden in a seed file
test('flags overdue invoices', async () => {
  await seedDatabaseFromFixture('invoices.sql');
  const overdue = await findOverdueInvoices();
  expect(overdue).toHaveLength(2);
});

// After — inputs are visible; the expected result is obvious
test('flags overdue invoices', async () => {
  await insertInvoice({ id: 1, dueDate: '2020-01-01', paid: false }); // overdue
  await insertInvoice({ id: 2, dueDate: '2099-01-01', paid: false }); // not yet due
  const overdue = await findOverdueInvoices();
  expect(overdue.map(i => i.id)).toEqual([1]);
});

For the file case, prefer a focused helper over a checked-in file:

const csv = makeSubscriberCsv(tmpFile, 'active@x.com', 'active@y.com', 'active@z.com');
// now "3 active" is justified by what you see in the test, not by a hidden file