Shipping AI code review that engineers actually trust

Every AI code review tool has the same honeymoon period. It ships, it's novel, teams enable it on every repo — and within two or three weeks, someone quietly disables it on the repos that matter most. We watched this happen inside our own early access cohort, and it forced us to rethink what "AI code review" was actually supposed to do.

Why review tools get muted

The failure mode wasn't accuracy in the way we expected. Our early model was right more often than it was wrong. The problem was confidence calibration: it flagged a genuine null-pointer risk with the same tone and urgency as a stylistic nit about variable naming. Engineers can tolerate a false positive here and there. What they can't tolerate is not being able to tell, at a glance, which comments are worth their attention and which aren't.

The second failure mode was worse: silence dressed up as approval. A comment-free review reads as "this is safe," but our model was frequently silent simply because it hadn't reasoned deeply enough about a change — not because the change was actually safe. That gap between perceived and actual confidence is what erodes trust fastest, and it erodes it permanently. Once an engineer catches a tool being confidently wrong, they stop reading its output altogether.

What changed the outcome

We rebuilt review generation around three constraints instead of one. First, every comment now carries an explicit confidence band, and low-confidence findings are collapsed by default instead of competing for attention with high-confidence ones. Second, we stopped generating comments from diff context alone — the model now walks the call graph for anything it flags, so a comment about an unguarded field access is backed by the three call sites that could actually trigger it, not a guess based on the line in isolation.

Third, and this mattered more than either of the above: we made disagreement cheap. A single click marks a comment as "not applicable here," and that signal feeds back into how aggressively that pattern gets flagged for that specific repository going forward. Trust isn't built by being right the first time. It's built by visibly getting better when you're told you're wrong.

What the numbers looked like after

Six weeks after the rework, comment-acceptance rate across our design-partner repos went from 31% to 68%. More importantly, the number of repos that had review fully disabled dropped from 22% to under 3% — and almost all of those remaining cases were repos with pre-existing linting setups that made our comments genuinely redundant, not repos where engineers didn't trust the tool.

Where this goes next

The next constraint we're working against is latency on large diffs — call-graph walking is not free, and a review that lands four minutes after a PR is opened is a review most engineers have already stopped waiting for. If you're running Forge on a monorepo and comment latency has been rough, that work is already in progress and should land for every workspace over the coming weeks.