How to Audit AI-Generated Code Before Shipping to Production

AI coding assistants now write a large share of the code that reaches production, and that code is not safe by default. The model that drafts your endpoint optimizes for plausible output, not correct or secure output — so it cheerfully reproduces the most common patterns in its training data, including the insecure ones. An audit before shipping is no longer a nice-to-have; it is the gate that catches what the model could not know about your system.

This is a practical checklist for reviewing AI-generated code before it goes live. It assumes you treat every generated diff as untrusted input until proven otherwise.

Why AI-generated code fails review

The failure is structural, not occasional. A large language model is a next-token predictor trained on enormous corpora of public code, much of which is itself outdated or vulnerable. It has no runtime context: it cannot see your trust boundaries, your auth model, or which inputs are attacker-controlled. So it emits code that compiles and "looks done" but breaks on the edge cases that matter.

The data backs this up. In the NYU study "Asleep at the Keyboard?" (IEEE S&P 2022), researchers built 89 security-relevant scenarios, generated 1,689 programs with GitHub Copilot, and found roughly 40% to be vulnerable, evaluated against MITRE's CWE Top 25. Stanford's "Do Users Write More Insecure Code with AI Assistants?" (ACM CCS 2023) found that participants with an AI assistant wrote significantly less secure code yet were more confident it was secure — 36% produced SQL-injection-vulnerable solutions versus 7% in the control group. Confidence is not a signal of safety.

The audit checklist

Run every AI-generated change through the same gate, regardless of how trivial it looks. The table maps each check to the underlying risk and the standard or tooling that backs it.

Treat broken access control as your highest-priority manual check: OWASP ranks it the number-one web application risk (A01:2021), present in some form in 94% of tested applications, with over 318,000 occurrences in OWASP's dataset. Static tools catch fewer of these than they catch injection, because authorization is application-specific logic.

The dependency trap: hallucinated packages

The most distinctively-AI risk is the hallucinated dependency. A study presented at USENIX Security 2025 analyzed 576,000 Python and JavaScript code samples across 16 LLMs and found 19.7% of recommended packages did not exist — over 205,000 unique fabricated names. Open-source models hallucinated far more (around 21.7%) than commercial ones (around 5.2%).

The danger is that these hallucinations are repeatable: re-running 500 triggering prompts ten times each, 43% of the fake package names appeared in every run. A predictable fake name is a squattable one. The Python Software Foundation's Seth Larson named this attack pattern "slopsquatting." It is not theoretical — security researcher Bar Lanyado of Lasso Security registered one hallucinated package name and recorded over 30,000 genuine downloads in three months. Name-based heuristics will not save you: per the same research, only about 13% of fakes were simple typos. You need Software Composition Analysis (SCA) plus pinned lockfiles resolving against a real registry.

FAQ

Can a SAST scanner replace human review of AI code? No. Static analysis is excellent for pattern-based flaws like injection and weak crypto, but it systematically misses authorization and business-logic bugs, which are application-specific. Use SAST as a mandatory gate, not a substitute for reading the diff.

Is AI-generated code measurably buggier than human code? A December 2025 analysis by CodeRabbit of 470 GitHub pull requests reported AI-co-authored changes carried about 10.83 issues per PR versus 6.45 for human-only PRs — roughly 1.7x more, with security issues disproportionately higher. Treat it as a directional signal, not a universal law.

What is the single most important check? Confirm every imported dependency actually exists and is pinned, then verify access-control logic by hand. Those are the two areas where AI fails in ways automated linting does not reliably catch.

Why is the model so confident when the code is wrong? It is trained to produce plausible text, and security correctness is not directly observable in most training signals. Plausibility and correctness are different objectives, which is exactly why human review is the gate.

Bottom line

AI-generated code is untrusted input. Gate it with SCA and pinned lockfiles (against slopsquatting), SAST mapped to the OWASP Top 10 and CWE, secret scanning, and tests — then read the diff yourself, prioritizing access control. Author confidence, the model's or yours, is not evidence of safety; the Stanford finding that AI users were more confident and less secure is the whole reason the checklist exists.

Sources and further reading