Duplication Instead of Extraction.
AI assistants tend to paste a fresh, slightly-tweaked copy of existing logic instead of reusing or extracting a shared function, inflating duplicated code while refactoring quietly disappears.
##Signs and Symptoms
A reviewer recognizes this smell when an AI-authored diff adds a block that already exists somewhere in the repo, lightly renamed, instead of calling the function that's already there. The tell-tale shape is two or more near-identical bodies that differ only in a noun (user/order), an endpoint, or a constant — the kind of copy that a human would have factored out.
Common signals:
- New handlers/services that repeat the same validation, retry, error-mapping, or fetch boilerplate verbatim.
- Magic strings and config literals re-typed in each new block rather than referenced from one place.
- Each agent turn re-implements logic the previous turn already wrote (the agent has no memory that a helper exists).
git logshows consecutive AI commits adding lines but almost no "moved"/renamed lines — additions without consolidation.
// AI added this for the new route — but getUser already exists 30 lines above
async function getOrder(id: string) {
const res = await fetch(`${API}/orders/${id}`, { headers: authHeaders() });
if (!res.ok) throw new Error(`Request failed: ${res.status}`);
const json = await res.json();
if (!json?.data) throw new Error("Malformed response");
return json.data;
}
async function getUser(id: string) { // <-- 95% identical body
const res = await fetch(`${API}/users/${id}`, { headers: authHeaders() });
if (!res.ok) throw new Error(`Request failed: ${res.status}`);
const json = await res.json();
if (!json?.data) throw new Error("Malformed response");
return json.data;
}
This is the classic Duplicate Code smell, but produced systematically rather than by accident.
##Reasons for the Problem
Why models produce it
- Next-token locality. An LLM completes the current span from the most similar text in its context window. Reproducing a nearby, known-good block is the highest-probability continuation; "stop, go define a helper, then call it" is a longer, lower-probability detour that spans multiple files.
- No repo-wide model. The assistant rarely knows a suitable helper already exists outside its context. Without retrieval it can't reuse what it can't see, so it regenerates. Practitioners describe agents that "recreate similar logic from scratch for each new task," and Eno Reyes (Factory) frames code quality itself as the main predictor of whether AI helps or hurts a codebase.
- Per-turn amnesia + prompt scope. Each request is treated as a self-contained unit ("make this work"), so the agent over-fits a narrow, single-use solution instead of generalizing. OX Security names this Over-Specification (seen in 80–90% of AI code) and Avoidance of Refactors (80–90%): the model emits functional code for the immediate prompt but never consolidates.
- Sycophancy / least-friction. Models optimize for an output that visibly satisfies the ask without touching unrelated files. Duplicating is "safe" and local; refactoring risks breaking something the model can't see, so it avoids it.
- Tooling makes it cheap. As GitClear notes, tab-to-accept makes inserting a fresh block nearly free, removing the friction that used to push developers toward reuse.
Why it hurts
- Measured quality erosion. GitClear's 2025 analysis (211M changed lines, 2020–2024) found copy/pasted lines rose from 8.3% to 12.3% while "moved" (refactored) lines fell from ~24% to 9.5% — 2024 was the first year copy/pasted lines exceeded moved lines, and blocks of 5+ duplicated lines jumped ~8×.
- Correctness debt — "Bugs Déjà-Vu." OX Security found this in 70–80% of AI code: a bug fixed in one clone silently survives in all the others, so identical defects recur and each needs a redundant fix. Duplication turns one bug into N bugs.
- Maintainability & review load. Every behavior change must be found and edited in many places; diffs balloon with near-identical blocks that reviewers must diff line-by-line to confirm they're actually identical (and not subtly, dangerously different).
- Security surface. A vulnerable pattern (missing auth check, unescaped input) propagates into every copy, and a later hardening fix can miss clones — duplication multiplies the patch surface.
##Treatment
Review & prompting tactics
- Tell the model where to look first: "Before writing new code, search the repo for an existing helper that does this and reuse it; do not duplicate." Paste the relevant module so it's in context.
- Demand consolidation explicitly: "If two blocks differ only by parameters, extract one parameterized function (Extract Function) and call it from both."
- Make it run the tools: "Run the linter / jscpd / the duplication check and resolve any clone it reports before returning the diff." Agents self-correct well when given a deterministic signal.
- In review, treat a pasted-looking block as a prompt to grep: search for one distinctive line; if it already exists, send it back for extraction.
The refactor — Extract Function + parameterize
// after: one source of truth, called from both
async function getResource<T>(path: string): Promise<T> {
const res = await fetch(`${API}/${path}`, { headers: authHeaders() });
if (!res.ok) throw new Error(`Request failed: ${res.status}`);
const json = await res.json();
if (!json?.data) throw new Error("Malformed response");
return json.data as T;
}
const getOrder = (id: string) => getResource<Order>(`orders/${id}`);
const getUser = (id: string) => getResource<User>(`users/${id}`);
Now a fix to error handling or response parsing lands once. Where the duplicates differ by a step rather than a value, prefer Form Template Method or pass the varying step as a callback. The canonical moves here are Extract Function, Extract Variable/Constant (for re-typed literals), and Pull Up Method when clones live in sibling classes — all standard cures for Duplicate Code.
Guardrail, not just cleanup: wire a clone detector into CI with a low threshold so the next AI-generated duplicate fails the build instead of merging. Pair it with a one-line house rule in CLAUDE.md/AGENTS.md ("reuse existing helpers; never paste a near-duplicate") so the constraint rides in the model's context on every turn.
##Detected by
- jscpd copy/paste duplication (configurable min-tokens / min-lines threshold; supports --threshold gate in CI)
- PMD CPD (Copy/Paste Detector) — token-based clone detection across 30+ languages
- SonarQube / SonarSource S4144 — Methods should not have identical implementations
- SonarQube / SonarSource S1192 — String literals should not be duplicated
- SonarQube Duplicated blocks / duplicated_lines_density metric (built-in clone engine, fails quality gate over threshold)
- eslint-plugin-sonarjs sonarjs/no-identical-functions
- eslint-plugin-sonarjs sonarjs/no-duplicate-string