DEV Community

Debashish Ghosal
Debashish Ghosal

Posted on

Our Agent Gate Blocked Every Bad Change. So Everyone Started Rubber-Stamping It.

Our Agent Gate Blocked Every Bad Change. So Everyone Started Rubber-Stamping It.

We built a gate for agent work. It blocked bad changes beautifully. That was the problem. Nobody read it anymore.

This is the story of how a perfect control trained the whole team to approve on reflex, and what the fix actually was. Spoiler: it wasn't a better gate.

How a gate dies

The sequence is boring and inevitable.

First, the gate catches real problems. Blocked changes save real weekends. Trust in the gate grows.

Then, agents start producing most of your diff volume. Review queues fill up. Every approval is the same ritual: open diff, it's long, the gate is green, the tests passed, approve. Nobody says they're not reading. Everyone is speed-reading the summary, the checks passed, that's the entire review.

The gate didn't fail. It succeeded itself into ceremonial status. A control people trust blindly is a control that no longer exercises judgment, it just adds latency before the same yes.

The moment it gets noticed is always the same one. Three hundred lines of agent diff, the summary says tests passed, the finger is already moving toward approve. Then something feels wrong, a second look happens, and the second look finds it: a test the agent wrote for its own change, asserting the wrong thing, passing, because it was testing its own behavior. The gate was green. The gate was right, technically. The review was about to be wrong, and the review was a tired human.

The shape shows up in real release work: a green pipeline that lied, a check that silently did nothing while reporting success. The failure isn't in the tools. It's in the relationship between the tools and the tired human at the end of the chain.

The gate is a product surface

Once you accept that the human is part of the gate, the design problem changes completely. A gate isn't a checkpoint, it's an interface, and interfaces have UX whether you design it or not. So design it:

Choices, not proofreading. Instead of a wall of diff, the gate presents decisions: this changed files outside its scope, allow or reject? Each one is a real question a human can answer while tired.

Blast radius first. The gate leads with what the change touches: credentials, migrations, API surface. You judge danger before you judge elegance. Elegant agent code is cheap. An agent touching auth is the thing to slow down for.

Per-change, not per-batch. Four agent PRs in one approval is four chances to say yes when you meant to say no to one. One at a time, no matter how much the queue begs.

Force the failure into view. If a check is green, show me what it ran and what it covered. The summary saying "tests pass" is a claim. The artifact is the evidence, and the gate should surface the evidence by default.

Why this actually matters now

Agent volume makes this urgent, because the review bottleneck is the one everyone's feeling. Harness's 2026 survey put it at 81% more time spent on review work than a year ago, and it's the top tax on all this speed. The tool response to that has been more AI review, more automation, more gates.

But every automated layer you add without touching the human loop just accelerates the rubber stamp. The bottleneck was never the checking. It was the judgment, and judgment can't be outsourced to a system whose output you trust enough to stop reading.

There's an uncomfortable symmetry with the agents themselves. We ask if we can trust the model, tune its guardrails, gate its tools. Then a human skims its output at 11 p.m. and approves in nine seconds. Which one of those two is actually unguarded?

What I can't claim

Better gate UX is not a security control. It's an attention allocator. It raises the floor on tired mornings, it doesn't raise the ceiling of what a good reviewer catches. And I'll be honest about the failure mode I still don't have a fix for: the moment people learn the gate is good, they trust it, and trust is the thing you needed to keep them skeptical. Every fix for that is cultural, slow, and fragile.


We didn't need a smarter gate. We needed a gate a human could disagree with at 8 a.m. That's a UX problem, and it was ours, not the agent's.

How many seconds do you actually spend on an agent PR before approving? Don't estimate. Time it.

Top comments (13)

Collapse
 
madhabcoder profile image
Nilamadhab Senapati •

"Succeeded itself into ceremonial status" is the best description of a green checkmark I've read. The agent writing a test for its own change that asserts the wrong thing is the part that scares me most, because it looks like extra safety. Blast radius first makes a lot of sense to me. I'd much rather answer "this touched a migration, okay?" than scroll 300 lines hoping my eyes catch it.

Collapse
 
debashish_ghosal profile image
Debashish Ghosal •

"The agent writing a test for its own change that asserts the wrong thing … looks like extra safety" — that's the exact failure that made me stop trusting green on agent PRs. A test written by the same process that wrote the change proves consistency, not correctness; it's a mirror, not a referee. Blast-radius-first is my answer too: migrations and money paths get a human regardless of how green the diff is.

Collapse
 
madhabcoder profile image
Nilamadhab Senapati •

"A mirror, not a referee" is going in my notes. Migrations and money paths always getting a human, however green it looks, is the simplest rule I've heard that actually holds up.

Thread Thread
 
debashish_ghosal profile image
Debashish Ghosal •

Glad that line landed. Once a test is understood as a mirror, the review question writes itself: what would make this test fail? If the honest answer is "nothing in this change," the test is decoration with a green check on it. Migrations and money paths getting a human regardless of the color is the blast-radius-first rule doing its job, and it is the one part of the gate I would refuse to automate even on the good days.

Collapse
 
eventuallyconsistentcode profile image
John Reed •

"Don't estimate. Time it." is the line that got me, because I went and looked and I measure everything about the agent and nothing about me…

I build cairn, a work harness for coding agents, and I read your four fixes against it this morning. Scorecard, honestly: the evidence one I mostly have, a task can't close without the command it declared and what came back, and a missing one is a flag, not a green word. The other three I don't. The question I get asked before a push opens with "N commits to origin/main," which tells me how much there is and nothing about whether auth or a migration moved. The harness already works out what a diff touches, it uses that to pick which reviewers to seat, and then never tells the human. Blast radius computed and not shown is a pretty good definition of the problem you're describing.

Where I'd push back a little is per-change over per-batch. I went the other way on purpose, one question per gate and never a second ask, because the failure I was running from was twelve prompts in a row and a finger that learns to hit yes by the fourth. Too many asks and one ask that's too big end up at the same reflex. What I wrote down today is the middle: routine things stay in the one batch, and anything that trips a danger signal (credentials, a migration, public surface, the harness's own config) leaves the batch and gets asked alone, first, with a cap on how many of those a run can ask before it just stops and reports.

Your near miss, the test the agent wrote for its own change, I only catch by judgment, a reviewer seat that looks for tests that can't fail. So that's filed too: a test added in the same work as the code it covers isn't evidence until it's been run against the parent commit and seen to fail. If it passes there too, that's the finding.

And the timer. Seconds between the question showing up and the answer, next to the size of what was approved, reported and never blocking. It's a proxy and I know it, a fast yes from someone who watched the whole run is fine, but nine seconds on three hundred lines is at least a place to look. It doesn't fix the part you said you can't fix, people trusting a gate because it's good, it just makes the drift visible before the bad morning instead of after…

Collapse
 
debashish_ghosal profile image
Debashish Ghosal •

This is the best kind of reply the article could have gotten, a builder scoring their own harness against it honestly, so thank you for the scorecard first. "Blast radius computed and not shown" is a better definition of the problem than the one I managed to write: the harness knows, the human pays, and the interface between them is the whole failure. On the pushback, I think you win the point. My per-change rule was written against twelve prompts in a row training a yes-reflex, and your middle version, routine things batched, danger signals leaving the batch, a cap that stops and reports, handles the same failure without rebuilding the queue anxiety I was arguing against. I am keeping my rule for the danger slice and conceding the rest. The parent-commit test is the sharpest thing in your comment: a test added alongside the code it covers is not evidence until it has failed somewhere, and if it passes against the parent too, that finding is the review. Filed, with credit. The timer as reported-and-never-blocking matches what the article can honestly claim: it allocates attention, it enforces nothing. If cairn goes public I would like to read how the danger-signal cap gets tuned.

Collapse
 
mike-johnstone profile image
Mike Johnstone •

For a regression fix, failing on the parent is useful evidence. For a refactor, a new characterization test can legitimately pass on both versions. In that case I'd check that it fails on a deliberately broken variant, rather than treating a green parent run as a warning. The gate needs to know what the test is meant to prove before it can interpret that result.

Collapse
 
liorbuilds profile image
Lior •

A review gets faster to trust when it shows only what changed and why, in a few plain lines, instead of the whole diff. One habit that helps: ask the agent to end every change with "what could this break?" in one sentence. If that sentence is vague or empty, that is the signal to slow down, and it gives the human something specific to disagree with.

Collapse
 
debashish_ghosal profile image
Debashish Ghosal •

"Show only what changed and why, in plain lines" is the right read of the fix — the gate failed as a review surface, not as a blocker. I like your one-sentence "what could this break?" because it's cheap enough to actually get answered, and I'd add the second half: the reviewer's job is to disagree with that sentence, which gives the nine seconds a purpose.

Collapse
 
liorbuilds profile image
Lior •

Agreed on the second half. One way to make the disagreement a habit: the reviewer writes one line back, either "agree" or "this could also break X". Even a one-word "agree" counts as an answer, and it makes silent rubber-stamping visible in the history.

Thread Thread
 
debashish_ghosal profile image
Debashish Ghosal •

I like that — "even a one-word 'agree' counts as an answer" converts approval from a reflex into a decision, and the thread history becomes an audit trail of attention instead of a wall of green checkmarks. The tell I'd watch after adopting it is how the one-liners trend: weeks of pure "agree" is the same rubber stamp in a new costume. The value shows up the first time "this could also break X" costs someone ten minutes and saves a rollback.

Collapse
 
prpatel05 profile image
Pratik Patel •

The detail that stuck with me is the agent writing the test for its own change. I'd make that its own blast-radius line in the gate: if the same run touched the code and the assertion, say so up front and let the green check count for less. A test the agent wrote against its own behavior proves the change is consistent with itself, not that it's right.

I'd push a little on per-change review too. One PR at a time helps, but the reflex comes back by the third one in a row. Making the reviewer answer one question in their own words before approve is clickable, something like "what breaks if this is wrong," does more than any layout change. Nine seconds gets hard when you have to type a sentence.

Collapse
 
debashish_ghosal profile image
Debashish Ghosal •

Typing the sentence is a stronger version of my "answer in your own words" instinct, because it defeats skim-approve mechanically — nine seconds stops being enough the moment the button needs a sentence. And you're right that the agent-authored-test case deserves its own blast-radius line: if the same run touched the code and the assertion, the green check should count for less by rule, not by reviewer memory. Consistent-with-itself is a lower bar than right, and the gate should say so out loud.