12 Months of Vibe Coding in Production: The Technical Debt Report Nobody's Publishing

The Year We Shipped Everything

Sometime in mid-2025, our frontend team made a collective, somewhat unspoken decision: we were going to lean into AI-assisted development properly. Not dabbling with Copilot for autocomplete. Not using Claude to generate a boilerplate component here and there. We mean properly - Claude Code in the loop for architecture decisions, Cursor for everyday feature work, AI-generated tests as the first draft on every ticket, AI-assisted code review as a second pass before human eyes.

The results, measured by the metrics our engineering leadership cared about, were extraordinary. Feature velocity up roughly 3x. PR cycle time down from 4 days to under 18 hours. Ticket closure rate climbed every quarter. Two engineers who had been stuck maintaining a legacy payment integration finally shipped a complete rewrite in six weeks - something that had been estimated at six months just two years prior.

Nobody was complaining. The business was thrilled. The team was energised, genuinely so, because the boring scaffolding work - generating types from API schemas, writing repetitive test cases, translating Figma specs into styled components - had effectively been delegated. Engineers were spending more time on problems that actually required human judgement.

Then, around month eight, things started to feel strange. Not broken. Just strange. A pull request for what should have been a simple bug fix touched eleven files and introduced three new utility functions - two of which already existed elsewhere in the codebase under slightly different names. A senior engineer spent an afternoon trying to understand why the same API response was being normalised in four different places, all producing subtly different shapes. A new joiner told me, somewhat apologetically, that they had needed three days just to understand the data flow for a single feature.

The code was syntactically beautiful. Clean, typed, commented, tested. But reading it felt like reading a book where each chapter had been written by a different author who had only been given a one-paragraph summary of the previous chapter.

AI Writes Locally Coherent, Globally Incoherent Code

This is the core issue, and it took me embarrassingly long to name it clearly: AI models optimise for the code that is immediately in front of them. They are extraordinarily good at making a function correct, readable, and well-typed in isolation. They are not good at knowing that a semantically identical function already exists three directories away, written by a different engineer three months ago using a slightly different naming convention.

The result is a specific kind of structural decay that I have started calling parallel invention - the codebase grows not through deliberate extension of existing abstractions, but through repeated independent creation of similar solutions. Nobody is writing wrong code. Every individual contribution is reasonable. The decay is emergent at the system level.

What this actually looks like in a codebase

In our codebase, we found three independent implementations of "format a date for display" logic. Not three files using a shared formatDate utility - three separate implementations, each slightly different in how they handled timezone offsets and null values. All three were AI-generated. All three were technically correct for the context in which they were created. None of them knew about each other.

// src/features/billing/utils/formatDate.ts
export const formatDate = (date: Date | string) =>
  new Intl.DateTimeFormat('en-IN', { dateStyle: 'medium' }).format(new Date(date));

// src/features/dashboard/helpers/dates.ts
export const toDisplayDate = (isoString: string | null): string => {
  if (!isoString) return '--';
  return new Date(isoString).toLocaleDateString('en-IN', { day: '2-digit', month: 'short', year: 'numeric' });
};

// src/components/shared/DateLabel/index.tsx (inline, not extracted)
const formattedDate = useMemo(
  () => new Intl.DateTimeFormat('en-GB', { day: 'numeric', month: 'long', year: 'numeric' }).format(props.date),
  [props.date]
);

Three implementations. Different locale strings. Different null handling. Different output formats. A product decision to standardise date display across the app now requires a search, an audit, a consolidation, and a regression test pass across three features. What would have been a two-line change in a well-maintained codebase becomes a half-day project.

This pattern repeated across data transformers, error formatters, loading state handlers, analytics event builders, and form validation logic. Not because the engineers were careless. Because every AI-assisted session starts with a fresh context window and no knowledge of what the project already contains.

Your AI-Generated Tests Are Testing the AI, Not Your Requirements

This one is subtle and it took a production incident to make me see it clearly.

When you ask an AI to write tests for a function it just wrote, it will write tests that pass. This sounds obvious, even tautological. The issue is what those tests are actually verifying. In almost every case, AI-generated tests verify that the function behaves consistently with its own implementation - not that it behaves consistently with the business requirement that prompted the implementation.

Consider a function that calculates a discount. The requirement was: "apply a 15% discount for annual subscribers, capped at Rs. 2,000." The AI implements the function, then writes three tests: one that verifies 15% is applied, one that verifies the cap is respected, and one that verifies the function handles a null input. All three tests pass. All three tests are generated from the code, not from the requirement.

What the tests do not cover: what happens if the subscription type is missing entirely (not null - absent). What happens if the price is passed as a string instead of a number, because someone upstream changed the API response shape. What happens if the subscriber is annual but their subscription has lapsed - does the discount still apply? These are questions about the domain, and the AI does not know the domain. Only a human who attended the product meeting does.

The rule we eventually established: AI writes the test scaffolding. A human must write at least the happy-path test and at least one realistic failure-mode test from knowledge of the actual business logic. The AI's tests are treated as a starting point, not a deliverable.

AI Reaches for a Package Before It Considers the Platform

I have written about replacing npm packages with browser APIs before - specifically about how teams accumulate unnecessary dependencies because engineers default to package searches rather than MDN. AI assistants have, somewhat paradoxically, made this problem worse.

In our audit, we found that AI-generated code had introduced 23 new npm dependencies over 12 months. Seventeen of them were for functionality that the browser or Node.js runtime already provides natively - things like deep object equality checking, UUID generation, URL parameter parsing, and simple debounce utilities. The AI was reaching for packages not because the platform lacked the capability, but because packages appear more frequently in its training data than the equivalent native implementations.

The more concerning finding was not the bundle size. It was the security exposure. Four of the 23 new packages received security advisories within six months of being added. Two were for moderate-severity vulnerabilities. One was for a supply chain incident - a maintainer's npm account had been compromised and a malicious version was briefly published before being taken down. We happened to have version-pinned with a lockfile, so we were not affected, but the fact that three of those packages had a combined weekly download count of fewer than 80,000 - not enough for meaningful community scrutiny - should have been a red flag.

The AI Doesn't Remember the Architectural Decisions You Made Last Sprint

Every AI coding session starts with a blank slate. The model you opened at 9am does not remember the architecture decision your team made at last Monday's sync, the naming convention you agreed on in that long Slack thread three weeks ago, or the fact that you decided - with good reason - that all server state should go through React Query and no component should hold server data in local state.

This creates what I have been calling convention drift. Each individual session produces code that is internally consistent with whatever context was provided in that session. Across sessions, without deliberate effort to anchor the AI to the team's established patterns, the code drifts. Not randomly - it drifts toward whatever the model considers idiomatic based on its training data, which may be a year or more out of date relative to your current stack.

In our codebase, we ended up with React Query used correctly in 70% of the codebase and bypassed in 30% - not because engineers disagreed with the pattern, but because some AI sessions produced code that used direct useEffect-plus-useState for data fetching, and that code passed review because it worked, it was clean, and the reviewer did not always notice the convention violation in a large PR.

The fix: a project context file the AI reads on every session

The most effective single intervention we made was creating a CLAUDE.md file in the project root - a structured document listing the team's architectural conventions, the approved patterns for common tasks, the packages that are explicitly forbidden and their approved alternatives, and the current sprint's focus areas. Every AI session begins by reading this file. Convention drift dropped measurably within two sprints.

# Server State
ALWAYS use React Query (useQuery, useMutation) for any data that comes from the API.
NEVER use useState + useEffect for data fetching. This pattern is banned.

# Date Handling
ALWAYS import from '@/utils/date' for date formatting. Do not inline Intl.DateTimeFormat calls.
The locale is 'en-IN'. Do not use 'en-GB' or 'en-US'.

# Forbidden Packages (use native alternative)
lodash          -> Use native Array/Object methods
uuid            -> Use crypto.randomUUID()
qs              -> Use URLSearchParams
axios           -> Use our internal apiFetch wrapper in '@/lib/api'

# Error Handling
All API errors must be handled via the ErrorBoundary in '@/components/ErrorBoundary'.
Do not try/catch API calls in components. Use React Query's onError callbacks.

It sounds almost embarrassingly simple. But the absence of this document was responsible for a significant fraction of the consistency problems we accumulated. The AI is as disciplined as the context you give it.

AI Review Plus AI Generation Equals a Sophisticated Echo Chamber

Many teams - ours included for a period - use AI-assisted code review as a second pass after human review. The logic is sound: AI is good at catching security issues, missing edge cases, performance anti-patterns. It is a useful tool. The problem emerges when the code being reviewed was also AI-generated.

When an AI model reviews AI-generated code, it is, to a first approximation, reviewing code that was written in a similar style to code it has seen many times in its training data. It will confidently approve patterns it has learned are standard, even when those patterns are wrong for your specific context. It will not flag the new formatDate function as a duplicate of the existing one because it has no knowledge of what already exists in your codebase - unless you explicitly provide that context. It will not flag the direct useEffect data fetching because useEffect-plus-fetch is a pattern that appears thousands of times in its training data and is technically valid JavaScript.

We observed a specific failure mode we started calling the legitimacy laundering effect: a PR that had received an AI review stamp felt more rigorously checked than one that had only received a human review, even when the AI review had been perfunctory. Reviewers were psychologically inclined to spend less time on code that had already been "checked," even though the AI check was missing the very class of context-dependent errors that human reviewers are actually good at finding.

The fix was procedural rather than technical. We stopped treating AI review as a quality gate and started treating it as a pre-check that surfaces obvious issues before human review begins. Critically, we made a rule: the AI review output is never shown to the human reviewer before they have done their own pass. No anchoring. The AI's findings are revealed after, as a supplementary check. This is counterintuitive but it meaningfully preserved the independent critical thinking that human review actually provides.

What the Codebase Audit Actually Showed

After month ten, I ran a structured audit of what had changed over the previous 12 months. Not a qualitative vibe check - a systematic crawl. Here is what we found:

1. Duplication rate had increased 4x. We measured this by running a code duplication analyser across the entire codebase, comparing snapshots from before and after the AI-first transition. The percentage of code that had at least one near-identical copy elsewhere in the codebase had gone from roughly 6% to 23%. Not copy-paste duplication - semantically identical logic written independently.
2. Average file length had increased by 40%. AI-generated code tends to be more verbose than experienced human-written code, partly because it adds comments, type annotations, and safety checks even when not strictly necessary. This is not inherently bad, but it means more code to read and maintain.
3. The dependency graph had become significantly more entangled. Measured by average module coupling score, the codebase was 35% more tightly coupled than 12 months prior. AI generates code that reaches for whatever is convenient, creating implicit dependencies between modules that share utilities without a clear ownership boundary.
4. Test coverage was up, but test quality was down. Line coverage had increased from 64% to 81%. But when we ran a mutation testing pass - a technique that deliberately introduces small bugs and checks whether the tests catch them - the mutation score had actually dropped. More tests, catching fewer real errors. A coverage number that tells a flattering but misleading story.
5. Time-to-understand for new joiners had increased. We tracked this informally by asking new engineers how long it took them to feel confident making changes to an unfamiliar module. The number had gone from "usually a day" to "usually two or three days." They consistently reported that the code was readable in isolation but hard to reason about as a system.

What We Did to Course-Correct

The audit happened around month ten. We spent months eleven and twelve making changes. Here is what actually moved the needle, and what did not.

What worked

The CLAUDE.md architectural context file was the single highest-leverage change. Once every AI session started with knowledge of the team's conventions, the convention drift rate dropped noticeably within two sprints. It requires ongoing maintenance - it is a living document, not a one-time setup - but the investment pays back quickly.

Pre-commit duplication checks added a simple automated step to the PR pipeline: before opening a PR, a script runs a duplication analysis on the changed files and adds a comment listing any near-duplicate code found elsewhere in the codebase. Engineers are not blocked from merging, but they are made aware. This simple visibility step reduced the rate of new parallel inventions significantly because engineers would ask the AI to refactor toward the existing utility rather than merge a duplicate.

Separating "what to build" from "how to build it" changed how we used AI in planning. We stopped letting AI make architectural suggestions in the same session that it was writing the implementation. Architectural questions - where does this belong, what abstraction should this use, which existing pattern should this follow - were addressed in a separate review step involving a human senior engineer before implementation began. This added maybe 30 minutes per feature but eliminated an entire category of structural problems.

What did not work

Asking AI to review its own code for convention compliance turned out to be largely ineffective. Without grounding in the actual state of the codebase, the model's suggestions were too generic. "Consider extracting this to a shared utility" is less useful than "this already exists at @/utils/date.formatDate."

Mandatory AI-generated test coverage thresholds actively backfired. When engineers knew they needed to hit a coverage number, they would prompt the AI to generate more tests until the number was satisfied. The tests were syntactically valid and coverage-maximising, but they were adding noise without adding signal. We removed the AI-generated test coverage threshold and replaced it with a mutation testing pass on critical business logic modules.

The Rules That Actually Worked, Without Slowing Us Down

I want to be specific here, because the general principle - "be more disciplined with AI tools" - is easy to say and useless to act on. These are the specific, concrete rules we established, along with an honest assessment of the overhead each one adds.

1. Maintain a CLAUDE.md / agent context file in every major module directory. Not just the root - each major feature directory should have its own context file listing the conventions specific to that domain. Overhead: about 20 minutes per new module to set up, 5 minutes per sprint to maintain.
2. Before implementing anything new, search the codebase first. Make "is there an existing utility for this?" the first message in every AI session, not an afterthought. Overhead: adds one exchange to each session. Saves refactoring time that runs into hours.
3. Write the sad path tests yourself. Let AI generate the happy path and the obvious edge cases. But every feature must have at least one test written by a human that covers a failure mode the AI could not have inferred from the implementation - a realistic upstream data shape change, a race condition, a boundary case from the product spec. Overhead: 15-30 minutes per ticket. Catches the class of bug that causes production incidents.
4. Ban AI from architectural decisions. Implementation: AI is excellent. Architecture: human judgement required. Where a module lives, what abstraction it uses, whether it should share state with another module - these are not questions to delegate. Overhead: zero, because these decisions should always have been made by a human.
5. Run human code review before AI code review, not after. Preserve the independent critical thinking that human reviewers provide. AI review is a supplement, not a replacement. Overhead: no net change in review time; the AI review step is moved in the process, not added to it.
6. Package additions require a brief justification in the PR description. One sentence: why this package, and why not the native alternative? This forces the engineer to have the conversation with themselves (or with the AI) before the package lands in the lockfile. Overhead: two minutes. Benefit: we eliminated all 17 unnecessary package additions that would have happened in the audit period.

Who Should Vibe Code, and Under What Conditions

After 12 months and one honest audit, here is where I actually stand on this.

AI-assisted development is genuinely transformative for teams that have strong architectural foundations and experienced engineers who can maintain structural discipline while delegating implementation to AI tools. For these teams, the productivity gains are real, durable, and - with the right guardrails - do not come at the cost of long-term codebase health. We are in that category, and we will not be going back.

It is significantly more problematic for teams where the senior engineers are the ones doing the most vibe coding, because those are the engineers who should be the architectural guardrails. If the person who would normally say "wait, we already have a utility for this" is instead prompting an AI to generate a new one, nobody is doing the cross-codebase coordination that every large frontend codebase depends on.

It is actively dangerous for early-stage codebases where the architecture has not been established yet. The most important months in a codebase's life are the first few, when the conventions, the abstractions, and the module boundaries are being set. Getting those wrong compounds in every subsequent commit. AI is extraordinarily fast at generating code in those months - which means it is extraordinarily fast at setting conventions in stone before anyone has had time to evaluate whether they were the right conventions.

The engineers I most want to flag this to are the mid-level developers who are two or three years into their careers and who have, in the last 18 months, dramatically accelerated their output using AI tools. The acceleration is real. The risk is that they are building intuitions about software architecture from AI-generated code rather than from reading and reasoning about well-designed human-authored systems. AI writes locally coherent code. Systems thinking is not a local property.

None of this means stop using the tools. It means use them in a way that keeps the structural thinking where it belongs - in the engineer's head, not delegated to a context window. Ship faster. Build more. Automate the tedious parts aggressively. But read the codebase the way you expect a new joiner to read it, at least once a quarter. The technical debt that AI assistance quietly creates is not the kind that shows up on dashboards. It shows up in the faces of the engineers trying to make sense of it six months later.