1. Start with an Operating Model, Not a Tool
An AI reviewer can inspect a diff quickly, compare it with nearby code and propose questions that a person might otherwise ask later. It can also misunderstand intent, miss behaviour outside its context window, repeat the same weak observation or present a plausible mistake with unwarranted certainty. Those properties make it useful as an input to review, but unsuitable as the authority that completes review.
The control model therefore starts with two accountable people:
- The human author of record owns the change, can explain its intent and accepts responsibility for the evidence attached to it.
- The human reviewer of record makes the approval decision, challenges assumptions and confirms that the required controls ran independently of the AI.
The same person should not fill both roles for a consequential change. AI may support either role, but it cannot become a substitute approver, an invisible co-author or a reason to weaken an existing separation of duties.
Treat every AI comment as a hypothesis. A useful finding identifies the affected code, explains the plausible failure mode and points to evidence that a human can reproduce. Eloquence, confidence scores and volume are not evidence.
This distinction matters because code review is more than defect detection. It transfers context, checks that a change fits the system's design, makes operational risk visible and records why the team accepted that risk. An AI tool should strengthen those functions without taking ownership away from the people who operate the software.
2. Classify the Change Before AI Reviews It
A single review policy is too permissive for sensitive code and too cumbersome for routine work. The author assigns a provisional risk tier when opening the change; the reviewer confirms or raises it before approval. When a change crosses tiers, the strictest applicable controls govern the whole change unless it is split into independently reviewable units.
| Tier | Typical changes | AI role | Required human control |
|---|---|---|---|
| Tier 1: routine | Documentation, isolated test maintenance, mechanical refactors and low-impact internal tooling | May inspect the scoped diff and suggest findings or patches | Named author and reviewer; normal repository checks; material AI use disclosed |
| Tier 2: consequential | Customer-facing behaviour, business rules, dependencies, data access, service interfaces and performance-sensitive paths | May analyse and suggest, but cannot approve, merge or waive a failed control | Domain reviewer; explicit test evidence; rollout and rollback plan where behaviour can affect users |
| Tier 3: sensitive | Authentication, authorisation, privacy, payments, pricing, secrets, cryptography, destructive migrations, shared infrastructure, supply-chain controls and safety-critical logic | Restricted to an explicitly approved use case and context; analysis-only may be the appropriate limit | Specialist ownership; independent review; strongest deterministic gates; staged release and an exercised stop control |
Risk is determined by consequence and reversibility, not by line count. A one-line authorisation error can be more serious than a large test refactor. Generated code does not automatically make a change high risk either; it changes the evidence the reviewer needs, while the code's function and blast radius still determine its tier.
Some activity sits outside every tier because it should not reach the model at all: secrets, production credentials, unapproved customer data, contractually restricted material and source code that an organisation has not authorised the provider to process. The safe default is denial until data handling, retention, residency, access and model-training terms are understood and approved.
3. Ten Guardrails for Accountable AI Code Review
- Keep a named human author and reviewer of record. The author must be able to explain the change without delegating that explanation to a model. The reviewer owns the decision to approve or request changes. Repository permissions and branch protection should preserve these responsibilities even when the AI service is unavailable.
- Assign the risk tier before accepting AI findings. The tier determines which context the tool may inspect, who must review the change and which release controls apply. If the tool discovers that a supposedly routine change touches a sensitive path, review pauses until the tier and reviewers are updated.
- Give the tool least privilege. Prefer read-only access to the smallest repository scope needed for the task. Do not grant production credentials, unrestricted organisation access, merge rights or write access merely because the integration supports them. Time-bound credentials and auditable service identities make access easier to reason about and revoke.
- Treat repository content as untrusted input. Comments, issue text, test fixtures, generated files and documentation can contain instructions that attempt to redirect an agent or expose information. System policy must outrank repository text; tools should use allowlisted actions and destinations; output must never be trusted simply because it quotes the codebase.
- Trace material AI use. The pull request should record where AI materially shaped code or review: the tool, the affected scope, accepted findings and the verification performed. Traceability is for diagnosis and learning, not surveillance. Teams should define “material” so trivial completion does not create useless paperwork.
- Require evidence for every material finding. A finding should cite the relevant lines, state the failure mode and describe a test, counterexample or deterministic signal that can confirm it. Unsupported claims are questions, not blockers. A model's confidence score must never override missing evidence.
- Suppress duplicate and style noise. Let formatters, linters and established policy checks own rules they can evaluate deterministically. Group repeated findings by root cause, deduplicate against existing review comments and apply team-specific severity thresholds. Review attention is finite; noisy automation spends it.
- Route sensitive paths to the right humans. Use code ownership and path rules to bring security, privacy, database, platform or domain specialists into Tier 3 review. AI may help frame a question, but it cannot provide the independence or accountability those reviewers supply.
- Verify independently and deterministically. Compile, type-check, lint, test and scan through controls that do not depend on the same model judging its own output. Add migration rehearsal, contract tests, static analysis or policy checks when the risk tier requires them. AI can propose a test, but the test must run in a trusted environment and its result must be captured.
- Preserve a human stop control. A designated owner must be able to disable AI comments, revoke its access and return the repository to the ordinary review path without blocking delivery. Tier 2 and Tier 3 releases also need a proportionate rollback or containment path. Automation is only under control when people can stop it cleanly.
These guardrails should be encoded where possible: repository settings for approval, code-owner rules for routing, identity policy for access, CI for verification and a pull-request template for traceability. A document that contradicts the working system is not a control.
4. The Review Workflow
The workflow keeps AI early enough to be useful and bounded enough to remain accountable.
- State intent and acceptance criteria. Before asking for review, the author explains the user or system outcome, important non-goals and the evidence that will show the change works.
- Assign the risk tier and reviewers. The pull-request template identifies Tier 1, 2 or 3, while path rules add any required domain specialists automatically.
- Prepare a bounded context package. Provide the diff, relevant interfaces, approved local guidance and failing or changed tests. Exclude unrelated repositories, secrets and production data.
- Run AI analysis as an advisory pass. Ask for specific failure modes—correctness, security, concurrency, data integrity, compatibility or operability—rather than a generic verdict. Disable patching when the tier permits analysis only.
- Triage findings before presenting them as blockers. Deduplicate comments, remove style issues owned by deterministic tools and distinguish evidenced defects from questions. The author should not have to disprove a stream of unsupported claims.
- Verify accepted findings independently. Reproduce the issue, add or adjust a test and run the appropriate trusted checks. When the finding is wrong, record enough feedback to suppress the same pattern without storing sensitive prompts.
- Complete human review. The reviewer of record examines intent, architecture, failure modes and verification. For Tier 2 and Tier 3, confirm rollout, observability and recovery before approval.
- Merge through the normal control path. AI cannot bypass required approvals, branch protection or failed gates. The human author remains accountable for the final diff, including generated code.
- Observe and learn. Watch the release signals appropriate to the change. Feed confirmed useful findings, false positives, escaped defects and rollback lessons into policy and tool configuration.
A strong review record lets another engineer answer four questions later: What did the change intend to do? What could have gone wrong? Which evidence supported approval? Who accepted the residual risk? AI may help assemble that record, but named people must stand behind it.
5. Measure Safety, Signal and Flow
Comment count and model acceptance rate are tempting because they are easy to collect. Neither shows whether review improved. Establish a baseline before enabling the tool, segment results by risk tier and read speed alongside quality and reviewer effort.
| Measure | What to examine | Why it matters |
|---|---|---|
| Finding precision | Confirmed material findings compared with dismissed, duplicate and style-only comments | Shows whether the tool earns scarce reviewer attention |
| Evidence quality | Findings with reproducible tests, counterexamples or deterministic signals | Separates actionable analysis from plausible prose |
| Safety and quality | Escaped defects, security findings, reverts, incidents and policy exceptions by tier | Prevents faster review from hiding worse outcomes |
| Review flow | Time to first useful feedback, waiting time, active review effort and time to merge | Locates where the workflow helps or merely moves work |
| Control health | Correct tiering and routing, deterministic gate coverage, access exceptions and stop-control tests | Checks whether the operating model works in practice |
| Human experience | Reviewer cognitive load, interruption, trust, opt-out reasons and perceived usefulness | Reveals friction that raw throughput cannot explain |
Targets should be chosen before the trial and should include stop conditions. For example, a team may pause expansion when sensitive files are misrouted, deterministic checks are bypassed or unsupported comments overwhelm useful findings. The exact threshold depends on the system's risk; the requirement is to decide it before enthusiasm can move the goalposts.
Review the misses as carefully as the wins. False negatives often expose missing context or an unsuitable task. False positives often reveal vague policy, duplicate rules or a prompt asking the model for too broad a verdict. An escaped defect does not automatically prove that AI caused it, but it should trigger a traceable examination of the authoring, review and release controls.
6. Roll Out by Evidence, Not Enthusiasm
A reversible rollout protects both the codebase and the credibility of the change.
- Baseline and threat-model. Document the existing review path, data boundaries, sensitive paths, branch controls and quality signals. Agree who owns the service and who can stop it.
- Run in shadow mode. Generate findings without publishing them into pull requests. Evaluate precision, evidence, duplication, data exposure and path routing against completed human reviews.
- Open an opt-in Tier 1 trial. Use a small set of repositories and volunteers. Keep normal approval rules, make feedback easy and remove rules that deterministic tools handle better.
- Expand Tier 1 only after its exit criteria pass. Confirm access boundaries, traceability, useful-signal quality and stop-control operation. Publish limitations alongside the supported use cases.
- Test Tier 2 as a separate decision. Add domain reviewers, stronger evidence and release controls. Do not infer safety for consequential changes from success on documentation or mechanical refactors.
- Keep Tier 3 restricted unless it earns explicit approval. Some sensitive paths may remain human-only. Where AI is allowed, narrow the task, record the decision and rehearse containment before depending on it.
Each expansion should be a reviewable policy change, not silent scope creep. Record the newly allowed repositories, actions, data classes and risk tiers; identify the approving owner; and retain a quick route back to the previous state. If the service fails, ordinary code review should continue.
The goal is not to maximise AI participation. It is to improve the quality and timeliness of engineering judgement without making accountability ambiguous.
Frequently Asked Questions
What is AI code review?
AI code review uses a model to analyse a proposed code change and offer findings, questions or patch suggestions. It is an advisory input to review. The human author still owns the change, the human reviewer still decides whether to approve it, and independent controls still determine whether required checks passed.
Should AI approve or merge code by default?
No. The safe default is that AI cannot approve, merge, waive a failed check or satisfy a required human review. A named human reviewer of record makes the approval decision through the repository's normal branch-protection and separation-of-duties controls.
Which changes need the strictest guardrails?
Tier 3 changes need the strictest treatment: authentication and authorisation, privacy, payments and pricing, secrets and cryptography, destructive data migrations, shared infrastructure, software-supply-chain controls, public interfaces and safety-critical or high-blast-radius logic. These changes need specialist ownership, independent review, deterministic verification and a controlled release and recovery path.
How do you reduce false positives and review noise?
Constrain the AI to named failure modes and relevant context, require evidence for material findings, suppress rules already covered by formatters or static analysis, group duplicates and feed confirmed dismissals back into configuration. Trial changes in shadow mode before publishing comments, and measure useful findings rather than total findings.
Do these guardrails make code review slower?
They add deliberate work where risk demands it, particularly during rollout and on sensitive changes. The relevant question is whether end-to-end delivery becomes safer and more effective, not whether the AI produces comments faster. Tier 1 automation may shorten routine feedback; Tier 3 should favour defensible evidence over superficial speed.
What AI use should be disclosed in a pull request?
Disclose material use: the tool, the affected scope, AI findings or generated code that shaped the final change, and the independent verification performed. Teams should define “material” in advance so traceability remains useful rather than becoming a log of trivial autocomplete. Never copy secrets or restricted prompts into the disclosure.