Originally published on tamiz.pro.
AI coding agents have taken the developer world by storm. They generate code, write tests, and refactor entire modules in seconds. Developers love them — surveys consistently show that a majority feel more productive and faster with these tools. But the first large-scale controlled study tells a different story. The METR study (2025) found that experienced open-source developers using AI coding agents completed tasks 19% slower than a control group without AI — yet they believed they were 20% faster. This is the AI coding paradox: agents make you measurably slower while feeling substantially faster. Why does this happen, and what does it mean for engineering teams?
The 19% Slower, 20% Faster Paradox
The METR study is the most rigorous evidence to date. Over 16 experienced developers, each with an average of 11 years of coding experience, were asked to implement small features and bug fixes in mature open-source repositories. Half used an AI coding agent (specifically, an early version of a leading agentic tool); half did not. The results were stark: the AI group took 19% longer to complete the tasks. Yet when asked how they thought they performed, the AI group estimated they had been 20% faster than they actually were — a 39-point gap between perception and reality.
This is not a fluke. The study is small but carefully controlled, and the participants were highly skilled. If anything, the slowdown might be even larger for less experienced developers, who lack the expertise to spot subtle errors that agents introduce. The paradox is real, and it has profound implications for how we measure developer productivity.
Why Perception Lies: The Illusion of Speed
The feeling of speed is not entirely fabricated. AI agents do accelerate the initial phase of coding. You type a prompt, and a few seconds later, a plausible implementation appears. This is exhilarating. It feels like you've crossed a huge distance in no time. But the feeling is based on the time to first draft, not the time to working, correct, reviewed code.
Several cognitive biases reinforce this illusion:
- Automation bias: We tend to trust automated systems more than manual processes. When the AI produces code, we assume it's correct because it's "intelligent". This reduces the scrutiny we apply, leading to more bugs that surface later.
- Progress illusion: The act of writing code (or having it written for you) creates a sense of forward momentum. We conflate lines of code with progress, even if those lines are wrong.
- Overconfidence: Because the agent handles the "hard part" of syntax and boilerplate, we feel we have mastered the task. This overconfidence masks the true cost of verification.
In reality, the AI has simply moved the work from writing to reading, reviewing, and debugging. And reading code — especially code you didn't write — is often slower than writing it yourself.
The Measurement Gap: Time-to-First-Draft vs. Time-to-Correct
The core issue is a mismatch in what we measure. When you ask a developer "how fast did you go?", they subconsciously answer based on the time to produce a draft. But the real productivity metric is the time to deliver a correct, integrated, and reviewed change. AI agents excel at the first; they fail at the second.
Consider a typical task: implement a new feature in a legacy codebase. With an AI agent:
- Draft generation: ~2 minutes (vs. 20 minutes to write manually).
- Review: You read the code, but because you didn't write it, you must understand the agent's logic. This takes 10 minutes (vs. 5 minutes if you wrote it yourself).
- Debugging: The agent's code compiles but has a subtle off-by-one error. You trace it, fix it, and run tests. This takes 30 minutes (vs. 5 minutes if you had written it correctly the first time).
- Integration: The agent's code conflicts with existing patterns. You refactor. Another 15 minutes.
Total with AI: 57 minutes. Total without: 30 minutes. The AI made the draft 10x faster but the total 1.9x slower. This is the paradox in a nutshell.
The Hidden Costs of Context Switching
AI agents don't just generate code; they generate context switching. When you delegate to an agent, you must shift your mental model from "I am writing this function" to "I am supervising a black box that writes this function". This is a fundamentally different cognitive mode, and switching between them is expensive.
Research on task-switching shows that context switching can consume up to 20% of a developer's productive time. With an AI agent, you switch contexts constantly: you write a prompt, wait, review the output, switch back to your mental model, spot an error, prompt again, and so on. Each switch carries a cognitive tax. The agent's speed doesn't pay for the tax.
Moreover, the agent's output is often plausible but wrong — it compiles, it looks reasonable, but it violates implicit assumptions about the codebase. Catching these violations requires deep understanding, which is exactly what the agent erodes. You can't trust the code, so you must re-verify everything, and re-verification is slower than writing.
When Agents Actually Help
It would be wrong to conclude that AI coding agents are useless. They shine in specific, low-stakes scenarios:
- Boilerplate code: Generating repetitive CRUD operations, serialization, or test scaffolding. The risk of subtle errors is low, and the speed gain is real.
- Unfamiliar languages: When you need to write a quick script in a language you don't know, an agent can get you 80% of the way there.
- Refactoring: Simple, mechanical renames or extraction can be safely automated.
- Test generation: Creating test cases for well-defined functions is a good fit, though the tests themselves need review.
But for complex, nuanced tasks — implementing a non-trivial algorithm, integrating with a legacy system, or fixing a subtle concurrency bug — the agent's weaknesses dominate. It lacks the deep context of the codebase, the understanding of architectural intent, and the ability to reason about edge cases. In these scenarios, the 19% slowdown is not just a statistic; it's a warning.
Implications for Engineering Teams
The paradox has serious consequences for how we manage and measure engineering performance:
- Don't measure productivity by feel. If your team reports feeling 20% faster, but pull request review time and bug rates increase, you're not actually faster. Use objective metrics: cycle time, change failure rate, review turnaround, and rework percentage.
- Beware of skill atrophy. If developers delegate all coding to agents, they lose the ability to write correct code themselves. This is a long-term risk that compounds.
- Reconsider the ROI of AI tooling. The METR study suggests that for many tasks, the cost of AI agents (licensing, review overhead, debugging) exceeds the benefit. Teams need to measure their own before/after data.
- Design workflows that account for the paradox. If you use agents, build in mandatory code review, enforce strict testing, and require developers to explain the agent's code before merging. This mitigates the hidden costs.
What to Do About It
So, should you stop using AI coding agents? No — but you should use them differently. Here are practical recommendations:
- Reserve agents for low-risk, high-repetition tasks (boilerplate, tests, migrations). Avoid them for core business logic.
- Never merge agent-generated code without a thorough human review. The review should focus on correctness, not style, because that's where the hidden bugs live.
- Track objective metrics before and after adopting agents. Measure cycle time, defect rate, and review effort. If your numbers worsen, scale back.
- Invest in your developers' core skills. The paradox exists because humans are still the bottleneck for verification. Keep those skills sharp.
- Set expectations honestly. Tell your team that AI agents will make them feel faster, but the real gains will come from careful integration, not blind delegation.
The AI coding paradox is not a failure of AI; it's a failure of measurement. Once you understand the gap between perception and reality, you can harness the speed of agents where it truly helps and avoid the trap where it hurts.
Frequently Asked Questions
Is the 19% slowdown universal?
No. The METR study is one controlled experiment with 16 developers on open-source projects. Results may vary by task type, codebase complexity, and tool version. However, the underlying mechanism — faster drafts, slower verification — is likely to apply broadly, especially for complex tasks.
Why do developers feel faster even when they're slower?
Because the feeling of speed comes from the time to first draft, not the time to correct code. The brain measures progress in terms of output (lines of code) rather than outcome (working, merged code). This is a cognitive bias, not a lie.
Should we abandon AI coding agents?
No. They are valuable tools for specific tasks. The key is to use them selectively and measure their impact objectively. Treat them like a power tool: great for cutting wood, terrible for carving fine details.
Top comments (0)