AI-Generated Code: Cleaner Style, More Real Bugs

Someone tracked 302,579 AI-authored commits into real repositories and watched what happened to them over the following year. The result splits in two directions at once, and most coverage has reported only one of them.

The code got tidier. The bugs got worse. Both of those are in the same dataset, and which one you care about depends on whether you are maintaining the software or running it.

The study

Debt Behind the AI Boom: A Large-Scale Empirical Study of AI-Generated Code in the Wild by Yue Liu, Ratnadira Widyasari, Yanjie Zhao, Ivana Clairine Irsan, Junkai Chen and David Lo.

The version 2 figures we quote here cover 302,579 verified AI-authored commits across 6,299 GitHub repositories with at least 100 stars, spanning GitHub Copilot, Claude, Cursor, Gemini and Devin, in Python, JavaScript and TypeScript. Commits were discovered from January 2024 to October 2025 and tracked through February 2026. Earlier and later versions of the paper report slightly different totals as the dataset was extended, so it is worth checking which version any figure comes from.

What makes it unusual is the tracking. Most studies test whether a model can produce correct code in a controlled setting. This one followed real code into real repositories and watched what happened to the problems afterwards.

The number that gets quoted

484,366 distinct issues across 27,677 commits, which is 9.1% of the total. Between 17.4% and 29.1% of commits from each assistant introduced at least one issue. 22.7% of those issues were still present at the latest version of the repository, and over 100,000 unresolved issues had accumulated by February 2026.

That is the version you will find in most coverage, and it is accurate. It is also only part of the result.

The number underneath it

The paper also measured what AI fixed, not only what it broke. Those two directions point opposite ways.

On style it roughly broke even: 439,817 smells fixed against 432,748 introduced. That is a margin of about 1.6% in observational data from public repositories, which is well inside the noise of the confounds in a study like this. Read it as "AI is not making style worse" rather than "AI is making style better."

On the things that determine whether your software actually works and is safe, it went backwards. The paper states plainly that AI "introduces about 1.5 times as many security issues as it fixes." On correctness it also introduced more than it resolved, though the paper does not attach a specific multiplier to that category, so we are not going to invent one.

Why this split is exactly the wrong way round

Consider what a code review actually catches.

Style problems are visible. A badly named variable, a duplicated block, a function doing four things: a reviewer sees these by reading, and a linter catches many automatically. They are also the least dangerous. Ugly code that works still works.

Correctness and security problems are mostly invisible on a read. An off-by-one in a boundary condition, an unvalidated input three layers down, a race that only appears under load. Reviewers miss these routinely in human code, and AI-generated code is harder to review because the reviewer did not build up the mental model of the logic by writing it.

So AI improves the category that review handles well, and degrades the category that review handles badly. The 22.7% survival rate is what that looks like a year later.

The tool spread is a buying decision

The range from 17.4% of commits introducing an issue with GitHub Copilot to 29.1% with Gemini is a substantial difference. One tool introduced a problem in roughly one commit in six; another in nearly one in three.

Two cautions before anyone treats that as a ranking. This is observational data from public repositories, not a controlled trial, so tool choice is tangled up with who uses each tool and what they use it for. Someone using an agentic tool for large refactors is not doing the same work as someone accepting inline completions. And these are fast-moving products; the versions measured are not the versions shipping now.

What is fair to take from it is that the spread is wide enough to be worth your own measurement, and that treating all AI coding assistants as interchangeable is not supported.

What we changed

We use AI assistance on client work. We are not going to pretend otherwise, and the productivity is real. What we changed is where the review effort goes.

We stopped spending review time on anything a linter already catches. That time moved to boundary conditions, input validation, error paths and anything touching authentication or payment.

We also treat any AI-generated code touching security as requiring a test that demonstrates the failure case, not just the success case. Generated code passes its happy path almost by construction. The question is what it does with bad input.

This is a version of the same argument we make in what breaks after you vibe-code it: the speed is genuine and the cost lands somewhere specific, so aim your attention at that place rather than distrusting the whole output.

If you had something built with AI assistance

• Ask what was reviewed and by whom. "The AI wrote it and it works" is not a review.
• Ask for tests covering failure cases, not only the intended path. This is where generated code is weakest.
• Check input validation on anything reachable from the internet, particularly forms and file uploads.
• Do not be reassured by clean, readable code. Readability is the category AI is good at and tells you nothing about correctness.
• Assume style linting is already handled and spend your budget on a security-focused review instead.

The last point is the one that saves money. Many audit proposals bill for a general code-quality pass, and on AI-assisted code that is largely paying someone to confirm what a free linter already knows.

If you had something built quickly with AI tools, run a static analysis tool over it this week. It is free, it takes an hour, and it will tell you whether you have a style problem or a security problem before you pay anyone to look. If it comes back ugly and you want the second half done properly, that is a fixed-scope review under custom software development. Get in touch.