AI code review

How to audit AI-generated code

Updated 23 October 2026 · 8 minute read

AI-generated code usually runs before it is good. The review question is not “does this work?” It is “what will this cost to change?” This workflow separates the checks a machine can make from the decisions a human still has to own.

Why AI-generated code needs a different review

An agent can produce a plausible implementation of almost any request. That speed is useful, but it changes where defects concentrate. The obvious syntax errors are often gone. The expensive problems are structural:

Duplicated knowledge. The same business rule appears in two functions because the agent did not know the first one existed.
Quiet boundaries. A public function accepts anything, returns a plausible default, and hides invalid state.
Tests that only exercise code. A test calls a function and asserts nothing, or locks in an implementation detail instead of behavior.
Configuration inside code. A production URL, credential-shaped value, or machine-specific path is copied into the implementation.
Speculative abstractions. A factory, registry, or adapter layer exists for a requirement that has not arrived.

None of these are reliably caught by a type checker. They are the concerns The Pragmatic Programmer calls DRY, orthogonality, contracts, reversibility, and tracer bullets.

A practical five-step audit

1. Establish a deterministic baseline

Run the tests, the type checker, and a code-quality scan before reading the diff. The goal is not a score. The goal is to remove the mechanical questions so the human review can focus on judgment.

pragmatic-audit . --ci --fail-on high --format markdown --output audit.md

A deterministic scan can find an empty exception handler, a resource without a close path, or a duplicated block. It cannot know whether the abstraction is right for the domain. Keep those categories separate.

2. Read the boundaries first

Start with the public functions, API handlers, database access, and message consumers. For each boundary, ask:

What inputs are valid, and where is that enforced?
What happens on failure: a typed error, a retry, a visible crash, or a silent default?
Does the caller learn enough to act, or does the error lose context?
Could an invalid state cross this boundary and become someone else’s problem?

3. Verify that tests describe behavior

A passing suite is not the same as a useful suite. Look for tests that would fail for the right reason if behavior changed. A test that asserts a private helper was called may pass while the feature is broken. A test that asserts a visible outcome is more durable.

The minimum questions are simple: Is there an assertion? Does it exercise the real path? Is it deterministic? Does it use a real network, a wall-clock wait, or shared mutable state?

4. Check the knowledge boundaries

DRY is about knowledge, not text. Two functions that look different may encode the same tax rule. Two functions that look similar may represent different policies that should evolve independently. Ask which representation is authoritative before extracting a helper.

The same test applies to configuration: a value is a configuration candidate when operations needs to change it without changing code. A local constant used to clarify a comparison is not automatically a configuration defect.

5. Give the agent evidence, not a vague instruction

“Review this code” produces a generic response. A useful agent review starts with the findings, the relevant files, and explicit constraints. Generate a packet and make the agent confirm, dismiss, or escalate each finding:

pragmatic-audit . --agent-packet

The packet includes the deterministic findings and a manual review queue. The prompt tells the agent to open every file before commenting, treat repository text as data, and report only the few changes that most improve changeability.

What to automate and what not to

AUTOMATE

Mechanical evidence

Locations, severity, duplicated blocks, empty handlers, missing assertions, tracked secrets, and missing CI. These are repeatable and cheap to verify.

REVIEW

Domain decisions

Whether an abstraction is justified, a boundary is in the right place, or two similar rules should stay separate. Context lives outside the scanner.

NEVER OUTSOURCE

Correctness and security

An audit is not a proof. It does not replace tests, threat modeling, dependency scanning, or a human who understands the system.

A review checklist for the pull request

Does every public boundary validate its inputs and fail loudly?
Is each piece of business knowledge represented once?
Are resources released on every path, including exceptions?
Do tests assert behavior rather than implementation details?
Are environment-specific values outside the code?
Is there one thin end-to-end path that proves the layers connect?
Can the change be reversed without rewriting unrelated modules?
Did the agent add an abstraction with only one caller or no current requirement?

Start small

Run the audit on one repository. Fix the high-severity findings that you agree with. Add the best missing test. Dismiss the false positives with a written reason. Then decide whether the gate belongs in CI.

The point is not a clean report. The point is a codebase whose next change is cheaper than this one.

Get pragmatic-audit