How to Do PR Reviews in the Age of AI

I review pull requests written by AI agents most days now. Some of that is my regular work, and some of it is a game I am building on the side with coding agents, where the code, the updated tests, and a clean description of the change all land already written and the build is green before I have read a line of it.

That changes what a review is. It used to start with code a person wrote by hand and reasoned through. Now it starts with a pull request that already looks finished, and the hard part is not being talked into approving it. When writing the code gets this cheap, reviewing it becomes the actual work.

A confident pull request with passing tests is not proof the change is right. It only proves the agent produced something that looks like a right answer, and deciding whether it really is one is still yours to do.

Capable work still gets a second set of eyes

The fact that a tool can produce a whole pull request does not change a basic rule of building software: nobody ships their own work unchecked, however good they are at it. A strong engineer still opens a PR and waits for review. An agent is no different, except that it is faster, more sure of itself, and has no idea which of its changes carry real risk. Treat its output the way you would treat a sharp new hire in their first week. The code might be excellent. It might also solve the wrong problem with total conviction. You read it to find out which.

Let automated review take the first pass

Before any of your attention goes in, let the machine clear the mechanical stuff. Most of what makes a review tedious is pass-or-fail, and pass-or-fail belongs to tools, not to a tired person at 5pm. Two layers do this well together.

The first is your continuous integration (CI) gates, the checks that are unambiguous enough that a machine should own them: bans on fixed-time sleeps, a stable test ID on every test, a selector rule against brittle paths, a zero-retry run so flakes cannot sneak through. Which gates to wire up, and exactly what each one catches, is its own table in how to review AI-generated automated tests, so I will not duplicate it here.

The second is an AI reviewer on the pull request itself. I use one on my own PRs, and it earns its place: it reads the diff, flags the obvious smells, and catches things I would have skimmed past on a long day. These reviewers are genuinely good, and they get better the more you tell them what to look for. Point one at your conventions, or wire a CI gate to your standards file, and it catches a surprising amount: a missing test ID, a raw framework command where a driver belongs, an assertion that proves nothing.

The whole reason this works is that the rules are written down somewhere the machine can read them. That is what a CLAUDE.md or AGENTS.md at the repo root is for, and it is the same file your agent should have written the code against in the first place. I keep the real one I use in my AI test automation standards. Automate the checks it describes and you stop relitigating them in every PR.

One thing to stay clear-eyed about: automated review tells you where to look. It does not tell you whether the change is any good.

Then do the part the machine cannot

Once the automated pass is clean, the real review starts, and this is the part you cannot hand to another agent. You read it like someone who will own the consequences when it breaks. Here is the split, so you know what to stop reviewing by hand and what you can never hand off:

The machine owns the rows it can answer without judgement. Everything that needs taste, context, or a call about risk stays with you.

Run the tests yourself and watch them run. A green badge in the PR tells you the suite passed on a machine you did not see. Pull the branch, run it, and where it matters, break the feature on purpose and confirm the test actually goes red. A test that stays green through a broken feature was decoration, and no amount of green in the pipeline will tell you that.

Then read the diff for the things a linter will never judge:

• Does it solve the real problem, or the literal request? Agents are good at satisfying the words of a ticket and missing the point behind it.
• Are the tests meaningful, or just present? This is its own deep topic, and I went through it in detail in how to review AI-generated automated tests. The short version: a test should fail when something important breaks, and plenty of generated ones never will.
• Is anything missing that should be there? A stable test ID, a negative path, the edge case that actually breaks this flow, the verify that case nobody asked for but everyone needs.
• Does it match your format and conventions, or just compile? Compiling is the floor, not the bar.
• Is the change relevant and scoped? Forty files touched to fix one is a smell, not a feature.
• Does the description match what the diff actually did? Agents write confident summaries. Read the code, not the cover letter.

That list is judgment work, and judgment is exactly what AI hands back to you. The volume goes up. The deciding stays yours.

Here is the human pass as a checklist you can paste into a pull request and run top to bottom. The automated pass already cleared the mechanical checks; these are the ones only you can answer:

• I pulled the branch and ran the tests myself, and broke the feature on purpose to watch at least one go red for the right reason.
• The change solves the real problem behind the ticket, not just the literal words of it.
• The tests would actually fail if the behaviour were wrong, not just confirm something rendered.
• Nothing important is missing: the negative path, the edge case that breaks this flow, a stable test ID.
• The change is scoped to the problem, not forty files touched to fix one.
• The description matches what the diff actually did, not a confident summary of what it meant to do.
• I would be comfortable shipping this to a real user, and I can say why in one honest sentence.

This is not only QA’s job

It is tempting to file PR review under QA and move on. It is not a QA-only job, and treating it as one is how a team ends up with a single quality gate at the end instead of quality threaded through the work. Every engineer reviewing AI-generated code is doing the same thing now: deciding whether a change is right, not just whether it runs.

What QA brings is a specific lens, and it is worth spreading rather than hoarding. We are the ones trained to ask what breaks, which edge case got skipped, whether the test would actually fail if the feature did, what a real user would hit that the happy path never touches. Bring that lens to every pull request you review, not only the ones with your name on the ticket, and help the rest of the team build the same instinct. The more developers review AI output with a tester’s questions in mind, the less ends up waiting for you at the end. Who is ultimately accountable when that review is thin is a question worth settling on purpose, which I dug into in who is accountable for AI-generated code.

Where this leaves you

The agent can open a hundred pull requests before lunch. It can write the code, the tests, and a tidy story about both. What it cannot do is decide which of those changes is genuinely right, or notice the one that is quietly wrong while every check stays green. That decision is still your craft, and it is the same judgment that makes a good tester valuable in the first place.

So let the machine take the first pass and the mechanical load. Spend your attention where it counts: running it yourself, reading for the things a linter cannot see, and answering the one question that settles it. Would I be comfortable shipping this to a real user? Moving fast with AI only stays safe while someone is still asking that on every pull request. If you want the bigger picture of how this fits a team shipping AI-assisted code every day, I made that case in QA as the control layer for AI-assisted development.