DEV Community

Cover image for Is Coding with AI Actually Faster and Cheaper?
Sylwia Laskowska
Sylwia Laskowska Community Curator

Posted on

Is Coding with AI Actually Faster and Cheaper?

Real data meets token-heavy coffee chat humor

How has AI actually changed the way we work? Are coding agents really making us remarkably faster? And what if, instead of answering with β€œwell, that's what I think,” we tried looking at some actual scientific research?

This is another article inspired by the upcoming AI Coding Summit in Berlin (and yes, I'll leave a link to my speaker badge at the end again, which I think might come with some discounts). I promised myself I wouldn't do this again, but, as usual, I've also signed up for three panel discussions. πŸ˜… One of them has a title very similar to this article.

I wanted some hard data for that discussion, so I dug into the research. You'll find links to all the studies at the end.

And since today hasn't exactly been a great day for me, and I'd much rather be somewhere else, I figured I'd cheer myself up the usual way. By writing something for you!

Are we actually coding faster?

Let's start with the first question. Has AI really made us faster? If someone suddenly stopped me in the street and demanded an answer, I'd probably say yes. In my experience, we write code much faster now.

But that improvement doesn't seem to translate into an equally dramatic increase in how much software we actually deliver. After all, there's code review, analysis, discussions, and so on and so on. Sure, things are a bit faster. Maybe a little less stressful, too. But I wouldn't say the entire development process has suddenly become much quicker.

Then again, maybe it's just me? Maybe the rest of the world is already shipping features at the speed of light?

Well, apparently not.

A study by economists from MIT and Wharton, revised in September 2026, suggests that my intuition might not be entirely wrong. The researchers analyzed data from more than 500,000 GitHub developers and estimated the following cumulative changes associated with autonomous coding agents:

Metric Estimated change
Coding activity (commits) +240%
Software releases +30%
Total app usage across the marketplaces studied No overall increase

These are observational estimates. Still, the gap between the jump in commits and the increase in releases is pretty striking.

Okay, so we're committing a lot more and shipping somewhat more often. In my own work, it sometimes just feels a little less stressful. But it's hardly the revolution some people make it out to be. Maybe things look different in brand-new projects. In large, established systems like the ones I work on, I haven't seen anything close to a tenfold improvement.

Maybe that'll change when someone invents those super-self-healing-autonomous agents. For now, I sometimes feel like things are getting more complicated. With everything that's been happening around AI security lately, everyone seems to be getting a little obsessed with it. Myself very much included. πŸ˜…

But are individual developers actually faster?

Okay, we have more commits. But are we actually coding faster? Again, if I had to guess, I'd say yes, but not that much faster.

The data is mixed.

Study Result What was measured
Cui et al., Management Science, 2026 +26.08% Number of completed tasks
METR, July 2025 19% slower Time to complete tasks
Science, January 2026 +3.6% Estimated increase in quarterly coding activity on GitHub
Borg et al., June 2026 30.7% less time Median reduction in implementation time in the observational phase

Once again, the results are reasonably close to what I expected. I'm not going to spend the next ten paragraphs going over the methodology. If you're curious, the original studies are linked below. They looked at different developers, tools, and tasks, and each percentage measures something different.

Several of these studies do point to gains, though. We're not all suddenly 10x developers. xD But there are real improvements.

METR is the odd one out here. It's from 2025, which, in the current AI world, practically makes it ancient history. But I love one detail about it.

Before the experiment, developers expected AI to make them about 24% faster. The actual result? They were 19% slower! And even after the experiment, they still thought AI had made them faster.

It shows how much we can fool ourselves until we actually measure things. I swear, this applies to practically every area of life. For example, I used to think I barely ate anything. πŸ˜‚

To be fair, METR published an update in February 2026. The researchers think newer tools are likely to be more helpful, but their follow-up couldn't reliably estimate the speedup because of who was willing to participate.

Okay, but is software development actually cheaper?

If someone woke me up at midnight and asked whether AI had made software development cheaper, I'd probably say no. At least not yet.

The 2025 DORA research and a follow-up published in March 2026 point to something many of us have written about. Some of the time we save writing code goes straight into auditing and verifying it. So the savings on implementation can shrink by the time the change gets to production. DORA describes where that work can shift, leaving the overall cost question open.

Yes, yes, I've heard about multi-agent teams where one agent writes the code, another reviews it, another tests it, and then it all goes into production. I'm sure that can work in some situations. I'd be very cautious about trying that trick in a mission-critical system. Financial software, for example. Good luck with that strategy πŸ˜‚πŸ€ž

What about the cost of tokens?

One last question. Are we spending all the money we save by writing code faster on tokens?

I have to admit, the answer surprised me! If I had to guess, I'd have said yes, especially if you blindly use the most expensive models for absolutely everything.

But the findings are more interesting than that! Two recent studies looked at the economics of AI-assisted software development. One was published in September 2026 by Potluri and colleagues, and the other in October 2026 by Steffen Herbold.

Both suggest that the effectiveness of an AI agent can matter more than the price of the tokens it consumes. In Potluri's analysis, models that solved more tasks produced substantial estimated savings, while less effective models ended up costing more in many task categories. A cheap model can get expensive pretty quickly if it keeps failing and you have to pay for another try.

Herbold simulated 432 scenarios for teams using a mix of developers and AI agents. He included salaries, token costs, supervision, bugs, and technical debt. Teams with highly capable agents were usually cheaper than teams of developers working without AI. The results for less capable agents were mixed. In the simulation, agent capability mattered more than token price because better results cut the cost of checking and fixing the work.

Herbold's results come from simulations, and Potluri's estimates come from benchmarks and economic modeling. So we have two ways of estimating when the savings might outweigh the token bill. The actual cost of building and maintaining commercial software is a much bigger question.

So, what's the verdict?

It looks like AI really can make coding faster and cheaper. Several studies show gains on specific tasks, and the economic models suggest that capable agents can save money even after you pay for the tokens.

We still have less evidence about whether the total cost of building, delivering, securing, and maintaining software has fallen across the industry.

So far, the research mostly lines up with what a lot of us seem to feel. Skynet may already be sharpening its weapons, but it hasn't conquered the world just yet. πŸ˜…

And what about you? Do these findings reflect your experience? Has AI genuinely made your team faster and cheaper, or does it look completely different where you work?

I'd love to hear your thoughts!


If you enjoyed this article, you can also follow me on LinkedIn.

And if you'd like access to free materials from AI Coding Summit, you can register using my speaker badge.

References

https://www.nber.org/papers/w35275
https://doi.org/10.1287/mnsc.2025.00535
https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
https://metr.org/blog/2026-02-24-uplift-update/
https://doi.org/10.1126/science.adz9311
https://doi.org/10.1007/s10664-026-10889-1
https://dora.dev/research/2025/dora-report/
https://dora.dev/insights/balancing-ai-tensions/
https://doi.org/10.13021/jssr2026.5687
https://arxiv.org/abs/2610.20394

Top comments (75)

Collapse
 
pengeszikra profile image
Peter Vivo •

Hyper shortly: yes, my number is around 100x.

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Hahaha, but 100x what? Faster coding or higher frustration levels? πŸ˜‚

Collapse
 
leob profile image
leob •

Serious, or joking? And (if serious) 100 times which metric, exactly?

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Hahaha, I'm pretty sure Peter is trolling us! πŸ˜‚

Thread Thread
 
pengeszikra profile image
Peter Vivo •

No I am not, 4h later I will reveal

Collapse
 
kenwalger profile image
Ken W Alger •

I think one of the most interesting distinctions here is between generating code faster and delivering verified behavior faster. Those aren't necessarily the same thing, and I'm glad to see the research starting to examine that difference.

Something I've encountered in my own experiments with AI-generated software is that the cost doesn't just shift from implementation to testing. It can shift further upstream into discovering and articulating the contracts the software is supposed to satisfy.

An agent might produce a working implementation in minutes, but establishing whether it handles an irreversible operation correctly, preserves an existing invariant, or behaves appropriately under failure conditions can take considerably longer. In some cases, the verification effort exceeds the original implementation effort.

I've also been experimenting with multi-model review panels, and the results have been interesting. More reviewers don't necessarily produce more verified defects, and agreement between reviewers doesn't establish correctness. Each additional finding potentially introduces another investigation.

That makes me wonder whether we're measuring productivity at the wrong boundary. Instead of commits, lines of code, or even completed tasks, perhaps we should be measuring the cost of getting from a requirement to independently verified behavior.

Of course, that's considerably harder to measure than counting commits. Which may be part of why we keep counting commits.

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

I completely agree! I actually had a perfect example of this at work recently. The code was delivered pretty quickly, but then came the review, fixes, unexpected issues, and all the work needed to get everything stable. By the time we were done, it felt like pretty much the same amount of work as usual. πŸ˜…

One of the studies I mentioned actually looked at the number of releases rather than just commits, which is definitely a step in the right direction. But even that isn't a particularly reliable measure. A release tells us very little about the complexity of the changes, their quality, or the actual value delivered to users.

And that's what makes this whole topic so difficult to study! I increasingly feel that conducting truly reliable productivity research in software development is a methodological nightmare. There are just so many variables: team experience, project complexity, technical debt, testing practices, and even what we consider a successful outcome.

So yes, measuring the journey from requirement to independently verified behavior sounds much closer to what actually matters. But how do we measure that reliably across hundreds of different projects and teams?

And perhaps that's exactly why we keep counting commits. They're easy to count, even if they don't tell us much about actual productivity! πŸ˜‚

Collapse
 
dannwaneri profile image
Daniel Nwaneri •

Thanks for this Sylwia. You made me realize that the DORA point was actually a real cost.

The effort gained from coding has been moved to testing (which is never measured).

Not also that I did not know the METR had been updated. I mentioned the 2025 results in my last article, so I am going to correct it asap.

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Thanks, Daniel! Yes, things are moving really fast in this space!

I also think it's pretty hard to measure all of this accurately, because companies and projects are so different. I'm sure experienced developers building products from scratch have a much easier time than those of us (myself included!) working with legacy systems. πŸ˜…

And I totally agree about testing! The effort required is often seriously underestimated. And I'm not even talking about E2E tests here. I strongly believe every feature should also be tested by an actual human!

Collapse
 
dannwaneri profile image
Daniel Nwaneri •

That's a good point re. legacy systems - lots of issues tend to get kicked down the road in a new project but always end up coming to the surface when you start working with existing code.

And, yes! Even the best tests (written by humans, to be sure) will never catch everything that an end-user will find when they interact with the feature. That's also where my testing ends.

Awesome thread, thanks! πŸ™‚ - Sylwia

Thread Thread
 
sylwia-lask profile image
Sylwia Laskowska •

Haha, exactly, Daniel! πŸ˜„ I remember how hard I once had to fight to get a manual tester on my team. My argument was that automated tests would never catch something being unintuitive to use. Eventually, the company agreed, we hired a fantastic guy, and once he started finding bugs, he quickly paid for himself! :D

Collapse
 
leob profile image
leob •

AI needs to start assisting us with reviews and testing as well, or else we'll never realize the promised productivity gains ...

Collapse
 
dannwaneri profile image
Daniel Nwaneri •

Agreed leob. I think it's a good idea to use AI for a first pass review, and indeed some teams do that. The tricky part is to ensure that the review itself is reviewed. I would suggest using AI for review when the review is pass/fail, like checking tests, scripts and so on, but also ensure a human is at the end of the process. That way there's an actual time saving, and not just a time shift.

Thread Thread
 
leob profile image
leob •

If AI spits out that much code, then it becomes too big a burden to review all of it manually - the promised productivity gains would just not materialize ...

So, let AI do some of the boring work, and try to find some of those needles in those haystacks - but at the end of the day, the 'human dev' (do we need a good word for that at some point? the "AI dev" versus the "human dev"?) will be the one to make the call: whether an issue raised by AI is "real" and is worth looking at!

As always, it's just a tool - it shouldn't be in charge, or completely going in "autopilot" mode ...

Thread Thread
 
dannwaneri profile image
Daniel Nwaneri •

Indeed. What's memorable about that sequence is the last invocation. So AI identifies the needles but the human determines the value.

Maybe we could coin a term like "the accountable dev", which is basically the responsible party for approval.

Then it would be a question about accountability, rather than humans vs. AI. Nice thread, leob πŸ™‚

Thread Thread
 
leob profile image
leob • • Edited

Yeah that's a great take - accountability is the dimension that really matters! AI can't be held accountable, the buck ultimately stops with a human, unless we'd elevate our AI agents to "personhood" (and make them 'liable' - maybe we could even fire them if they misbehave?) ;-)

Collapse
 
maame-codes profile image
Maame Afua A. P. Fordjour •

Really enjoyed this. I like that you showed the difference between writing code faster and shipping software faster. That is what I see too. AI helps me write the first version of a pipeline or a Terraform change much faster. But I still have to read every line, test it, and make sure it does what the ticket asks. One small mistake can cause big problems. So a lot of the time I save, I spend checking the code instead.

The METR study surprised me the most. People were slower but still felt faster. That is a bit scary, because it means we can’t always trust our own feeling about how much AI helps.

The part about token cost was new to me. It makes sense that a cheap model that keeps failing can cost more in the end. Thanks for putting all these studies in one place!

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Thanks, Maame! 😊 Yes, I think taking the time to actually read and understand code is incredibly important, especially for people just entering the industry. I honestly believe that this ability will become a real competitive advantage!

And exactly! The code itself might be written in minutes, but then come testing, code review, edge cases, and all those unexpected issues... and suddenly the whole process might not be that much faster after all! πŸ˜‚

Collapse
 
nyaomaru profile image
nyaomaru •

Really interesting summary! I found this very useful. 😸

I definitely feel faster when working individually with AI, but I’m less sure about team-level throughput. Individual tasks get done faster, but the whole delivery process doesn’t always feel dramatically faster. 😿

We recently hit a routing bug after a TanStack Router patch update changed microtask timing in a cross-SPA setup. AI completely missed the cause, even after I described the symptoms, and I eventually had to trace it back through the release changes myself. πŸ™€

So for mission-critical systems, I still think fully handing things over to AI is pretty risky. πŸ‘€πŸ‘€

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Oh, that's exactly how it looks in practice! πŸ˜‚ AI recently changed something in my code completely on its own, too. The new implementation actually looked cleaner than the original, so I thought, well, maybe it makes sense? Then I asked AI whether it was functionally equivalent to the previous implementation, and it said... no. XDDDD

And then it turned out the new version also introduced an XSS vulnerability! XDDDD

(Of course, it never even made it to a branch, let alone production. πŸ˜…)

Collapse
 
yuhaixia profile image
Yuhai Xia •

Great roundup of the research β€” the METR finding is the one that keeps nagging at me too. In my own work, AI clearly speeds up the first draft of a feature, but the time I 'save' often resurfaces in review and testing: generated code needs careful checking, and a subtle bug introduced by an agent can cost more to track down than writing the code by hand would have. The +240% commits vs +30% releases gap captures that well β€” output goes up, shipped value much less so. I now treat AI output as a draft I am responsible for, and I budget explicit review time for it. That framing has made the speed gains real instead of illusory, at least on my team.

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Oh, thanks for adding this! πŸ˜„ See, @leob? It's not that different for solo devs either! :D

Collapse
 
leob profile image
leob •

Yeah apparently! Maybe (as always) it's a matter of "it depends"? ;-)

Collapse
 
xulingfeng profile image
xulingfeng •

Hahaha this article couldn’t come at a better time! I’ll chime in on the cost side. I just found out I’m our company’s top token consumer on the AI relay this month. I bought some snacks for the colleague who manages token quotas, asking him to bump up my limit πŸ˜‚. From what I see on the dashboard, my token usage is almost equal to my monthly salary. It’s like I’ve got a clone of myself, hahaha!🀣

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Hahaha, I'm crying! πŸ˜‚πŸ˜‚ You know, sometimes there really are team members you'd love to clone. Turns out you just need to buy enough tokens! :D

BTW, if we worked together, you could borrow some from me. I use relatively few tokens because I spend most of my time in meetings. xDDD

Collapse
 
xulingfeng profile image
xulingfeng •

Hahaha! That’s such a fun trade‑off. All those meetings save your token budget, while my heavy usage keeps chasing my salary πŸ˜‚

Thread Thread
 
sylwia-lask profile image
Sylwia Laskowska •

Hahaha, maybe I should join your company and we could make a deal! :D But then the chocolates go to me, not the admin! πŸ˜‚

Thread Thread
 
xulingfeng profile image
xulingfeng •

Deal! All kinds of snacks will be fully supplied, in exchange for your token quota support πŸ˜‚

Collapse
 
leob profile image
leob •

"my token usage is almost equal to my monthly salary" - propose to your boss you'll reduce the token cost by 90% if he raises your salary by 50%?

Collapse
 
xulingfeng profile image
xulingfeng •

That’s such a brilliant negotiation idea! I might get fired the second I bring it up though πŸ˜‚

Thread Thread
 
leob profile image
leob •

Well if that's the case then it's not really brilliant ... ;-)

Collapse
 
vinmay profile image
Vinmay Nair •

Interesting findings, @sylwia-lask ! I wonder how these results would differ between greenfield and brownfield projects. AI might speed up development in both but existing dependencies, backward compatibility and integration overhead in brownfield systems could easily offset those gains. Would be interesting to see if the studies account for this distinction.

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Absolutely! I think legacy systems are a whole different beast! πŸ˜‚ And it's not just about technical debt, dependencies, or backward compatibility. It's also about the fact that changing something that's already in production and that users have grown accustomed to is much harder than changing something that's still being built.

With greenfield projects, you have much more freedom to experiment and make changes. With brownfield systems, even a seemingly small improvement can have unexpected consequences. So yes, I'd love to see more research exploring this distinction!

Collapse
 
iictp profile image
IICTP •

The METR detail is the one that stuck with me too β€” expecting 24% faster, measuring 19% slower, and still feeling faster is such a clean illustration of how hard this is to judge from inside the work. The Herbold finding is the one I'd love to see unpacked more: if agent capability matters more than token price because better results cut the cost of checking and fixing, does that change how you'd pick a model for a team β€” pay for the stronger one and spend the savings on review?

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Exactly, those are really good questions! And I imagine measuring all this properly would be a methodological nightmare, especially since so much depends on the company and the project. In new projects, AI probably speeds things up dramatically, but in large legacy systems, the difference might not be nearly as impressive.

Collapse
 
nour_dude_314 profile image
nourdude •

faster, cheaper, but generally worse in terms of how functional every line of code is
(and worse in terms of the fact that trying to make ai fix its own bug is like trying to destroy a small pebble with a wrecking ball. sure the pebble may or may not be gone but now the whole apartment floor is gone)

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

You know what? I actually read some studies on this too, but I didn't include them in the article because it was already getting so long. πŸ˜… And interestingly, they suggested that it's not at all obvious that AI-generated code is significantly worse than human-written code!

Some comments may only be visible to logged-in visitors. Sign in to view all comments.