Agentic code review, or why code review is the highest-leverage place to focus right now
Code review burden is a factor of:
- how much does the reviewer need to understand about the code being reviewed in order to provide a good evaluation
- how bad is it if we underestimate the context required
Agents solve a lot of this, or they bring the bar down for the effort required to have code reviewed.
Anthropic’s Code Review reports under 1% of its findings marked incorrect by their engineers, and the figure I would actually show a manager: it raised their internal rate of PRs receiving a substantive review from 16% to 54%. The long tail of changes that used to get a glance and an approval now gets read by something.
The case for running multiple agents as code reviewers is that each agent will use different context and lenses.
The most useful result I have seen this year is not from a vendor. An engineer ran four reviewers in parallel, CodeRabbit, Sentry Seer, Greptile and Cursor BugBot, across 146 real PRs and 679 findings over three and a half weeks:
Of 617 distinct flagged locations, 93.4% were caught by exactly one of the four tools. 6% by two. Almost none by three. None at all by all four.
…
In practice: do not agonize over the single best tool, there isn’t one.