SI Lets Coders Write 25 Times More Code. They Ship Barely More Software
Economists tracked more than 500,000 developers. SI tools sent their coding through the roof, but finished software rose only about 30%, and app stores filled up without gaining users.

Every week someone on 𝕏 posts a screenshot of an SI agent spitting out a whole app in an afternoon. The message is always the same: programmers are about to be ten times faster, and the software flood is coming. A study of more than half a million real developers says the first part is true and the second part, so far, is not.
The paper is called “Writing Code vs. Shipping Code”, a National Bureau of Economic Research working paper by MIT economists Mert Demirer and Liyuan Yang and the Wharton School’s Leon Musolff. It first circulated in May and was revised in September, and this week it is making the rounds again among developers and SI watchers, because it answers the question every boss with a coding-tool budget is asking. Do these tools actually produce more finished software?
The short answer: they produce a mountain of activity and a molehill of results.
Twenty-five times the code, 30% more releases
The researchers tracked more than 500,000 GitHub developers and paired public GitHub records with usage data on when each person started using SI tools. They sorted the tools into three generations. First came autocomplete, like the original GitHub Copilot, which suggests the next line as you type. Then came what they call sync agents, like Claude Code running on your own machine, which you direct in real time. The newest are async agents, like OpenAI’s Codex running in the cloud, which go off and do a task while you do something else.
To keep the comparison fair, each developer who adopted a tool was matched with a similarly active developer from the same week a year earlier, before anyone had these tools. That sidesteps the problem of “non-users” secretly using SI on the side.
On the raw activity measure, the tools are a rocket. Commits, the saved batches of changes programmers make, rose about 30% with autocomplete, about 180% once developers added sync agents, and about 240% once they added async agents on top. The effect was biggest for less active developers, the people who were coding a little and suddenly code a lot.
Then the numbers fall off a cliff as you move closer to finished software. The paper’s Figure 1 shows developers using all three kinds of tools writing about 25.5 times as many lines of code. They touch five times as many files and open 2.7 times as many pull requests, the formal proposals to merge code into a project. They work on 1.8 times as many projects. But actual software releases rise only about 1.3 times, roughly 30%.

Sync agents alone show the same shape. Lines of code jumped 958%. Pull requests rose 86%. Releases rose 20%. Even humble autocomplete followed the pattern: 234% more lines, 30% more commits, 9% more releases.
Where all that extra code goes
The authors tested two explanations. One is that each release simply got bigger, so the work shows up in fatter updates rather than more of them. That turned out to explain only a small slice of the gap.
The bigger answer is experimentation, and a lot of it ends up in the trash. After adopting sync agents, developers added 7.0 times as many lines of code but deleted 12.2 times as many. They created 79% more branches, the side copies used to try out a feature, and many of those ideas were later dropped. Across GitHub, the share of new projects that get no further work after their first month has climbed from 60% in 2021 to 69%.
In other words, SI makes it cheap to start things. Finishing them still runs through a person who has to review the code, test it, fix what breaks and decide it’s good enough to put in front of users. The economists call this the “weak link” problem: a chain moves only as fast as its slowest link, and right now the slow link is human judgment. Their model puts a number on how hard it is to swap SI output in for that human work, an elasticity of substitution of 0.23, which in plain English means the two are strong partners, not easy substitutes.
More apps, same number of users
The most striking part of the paper goes beyond GitHub. The researchers pulled data from four of the biggest software marketplaces: Apple’s App Store, the Google Play Store, the Chrome Web Store and SourceForge. Since early 2025, the number of new apps showing up each month has roughly doubled on average.
Total use of those new apps has not gone up in any of the four. People aren’t downloading or rating more new software. They’re just spread across more of it. The authors checked whether shoppers were at least finding niche apps that suited them better, and the evidence didn’t support that either.
So the store shelves are fuller, and the customers are the same size crowd they were before.
What it means for coders and the companies paying them
This is good news for anyone who writes software for a living and has been told SI will make them obsolete by next year. The study found that human effort is the piece the tools can’t easily replace. Review, testing and release decisions are where the value gets locked in, and those still need a person who knows what good looks like.
It is a warning for companies that measure SI by lines of code or commits. Those numbers are the easiest to juice and the least connected to what customers get. A manager who sees a 240% jump in commits and expects a 240% jump in product is going to be disappointed.
The authors are careful about the limits. Their data covers open-source projects and app stores, not the private code inside big companies. They cite separate research by Chen and Stratton finding the same pattern in corporate software, with even smaller gains, which makes sense since big companies have more layers of sign-off. They also note that coding is where SI is most advanced, so other jobs may see the gap even sooner. The paper has not been peer reviewed, as is standard for NBER working papers.
Their own conclusion is blunt. Large task-level gains, they write, “have therefore translated only partially into shipped and used software thus far.” The tools have gotten very good at the part of the job that was never the hardest. The next leap won’t come from writing code faster. It will come from whoever figures out how to help people check, finish and ship it.
Sources
- National Bureau of Economic Research — Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools, Working Paper 35275 (May 2026, revised September 2026)
- National Bureau of Economic Research — Full paper (PDF), Demirer, Musolff and Yang
- National Bureau of Economic Research — Mert Demirer, author page (MIT economics, NBER)