FEATURED POST

What AI coding tools change in a software budget: What the AI developer productivity research measured, and which project phases actually get cheaper

In 2026 almost every software proposal mentions AI. Delivery will be faster, teams smaller and the budget lower, sometimes by half. Before you plan next year’s IT spend around those promises, it is worth looking at what has actually been measured. Controlled studies published since 2023 put the effect of AI coding tools on developer speed anywhere between 19% slower and 56% faster. Both numbers are real. They describe different kinds of work, and that difference matters more for your budget than any vendor’s headline figure.

This article sets out what the research measured, why the results disagree, where the time in a software project actually goes and what that means for a budget, phase by phase. The short version: AI makes writing code cheaper. It does much less for everything around the code, from deciding what to build to testing, moving data and getting the result accepted, and Bain’s research puts that at 65–75% of the time from idea to launch.

What the controlled studies measured

Six sources come up in almost every discussion. They differ in who took part, what work they did and how the effect was measured, so read the last column before the third.

Study	Who and what	Measured effect	Keep in mind
Peng et al., 2023 (GitHub Copilot)	95 freelance developers; implement an HTTP server in JavaScript from scratch	Task finished 55.8% faster	Small, self-contained new code; 95% confidence interval 21–89%
Google, 2024	96 Google engineers; add a logging service to existing code (10 files, 474 lines)	About 21% less time	Not statistically significant after controls (p = 0.086); Google’s internal tools
METR, 2025	16 experienced maintainers; 246 real issues in their own mature repositories (on average 22,000+ GitHub stars and 1M+ lines of code)	19% slower with AI	The developers expected to be 24% faster and afterwards still believed they had been 20% faster
METR follow-up, 2026	57 developers, 800+ tasks	4–18% faster	Confidence intervals include zero; 30–50% of developers skipped tasks they would not do without AI, so METR calls the data an unreliable signal and expects the real speed-up to be larger, by an unknown amount
DORA, 2024	Survey of 39,000+ professionals; modelled effect of a 25% increase in AI adoption	Documentation quality +7.5%, code quality +3.4%; delivery throughput −1.5%, delivery stability −7.2%	Survey-based model estimates, not timed tasks
DORA, 2025	About 5,000 respondents; 90% use AI at work, for a median of two hours a day	AI is now associated with higher delivery throughput, but still with more instability	DORA describes AI as an amplifier of how an organisation already works

Two more sources explain much of the spread. Stanford researchers, presenting data on more than 100,000 developers in over 600 companies, estimate an average net gain of 15–20% once rework and bug fixing are subtracted. In McKinsey’s 2023 lab study, developers documented code in about half the time, wrote new code in nearly half the time and refactored in nearly two-thirds of the time, but the savings shrank to less than 10% on tasks they rated as highly complex.

Why the results disagree

Three things separate the 55.8% from the minus 19%.

New code or existing code. Writing a small program from scratch is the task AI assistants handle best. Changing a large system is the task they help with least. The Stanford breakdown, which its authors describe as indicative (136 teams in 27 companies), shows how steep the gradient is:

Type of work	Average productivity gain
New code, low complexity	30–40%
New code, high complexity	10–15%
Existing code, low complexity	15–20%
Existing code, high complexity	0–10%

The same research found that in low-popularity languages such as COBOL, AI helps little even with simple tasks and can reduce productivity on complex ones.

Who does the work. METR’s participants maintained large, mature repositories they knew in depth. That is the situation in which an assistant has the least to add and its suggestions take the most effort to check. McKinsey saw the other end of the scale: developers with less than a year of experience sometimes took 7–10% longer with AI.

Speed of writing versus speed of delivery. Code written faster still has to be reviewed, tested, integrated and supported. DORA’s 2024 model links more AI adoption to lower delivery stability, and the 2025 report still finds more instability even as throughput improves. Bain adds that the time saved often isn’t redirected towards higher-value work.

One more warning comes from METR: developers who were measurably slower still believed they had been faster. Speed-ups reported by the people using the tools, which many vendor case studies rely on, are not evidence of a shorter project.

Where the time in a software project goes

Bain’s 2025 Technology Report puts writing and testing code at about 25–35% of the time from initial idea to product launch. The rest goes to deciding what to build, designing it, integrating it, moving data, getting users to accept it and putting it into production. Bain also finds that teams using AI assistants see productivity gains of 10–15%, rising to 25–30% only when the delivery process itself changes.

Put those numbers into a budget. The example below is an illustration, not data from a specific project. Take a modernisation budgeted at 1,000 person-days:

writing and testing code is 25–35% of the effort, so 250–350 person-days;
the AI gain on that share ranges from 0–10% for complex changes to existing code to 10–15% on average;
the saving is therefore between zero and about 50 person-days, at most about 5% of the budget.

Now assume that one module in five turns out to be used by nobody. Removing those modules before the project starts takes them out of every phase: analysis, design, code, tests, data migration, acceptance and the years of maintenance that follow.

Lever	Phases it affects	Saving on 1,000 person-days
AI coding assistants	Writing and testing code	0–50 person-days
Not rebuilding the 20% of modules nobody uses	Every phase, including maintenance after go-live	Up to 200 person-days

The 20% is an assumption for the example. In your system it may be higher or lower, and it can be measured before anyone estimates the work: Oracle Database and Oracle APEX already record most of what you need to know about which modules, reports and tables are used.

Phase by phase: what AI changes and what stays expensive

Phase	Where AI tools help	What stays expensive
Scope and requirements	Summarising documents, drafting user stories, turning a requirements document into a clickable prototype	Deciding what not to build. No model knows which features your users rely on unless you give it usage data.
Design and architecture	Exploring options, drafting diagrams and decision records	Trade-offs that depend on your constraints: regulation, data residency, team skills, licences
Writing code	Large gains on new, self-contained code (55.8% in the Copilot experiment)	Complex changes to large existing systems: 0–10% in the Stanford data, and less help in low-popularity languages
Documentation	Quality +7.5% per 25% more AI adoption (DORA); documentation in about half the time (McKinsey)	Keeping it accurate as the system changes
Review, testing and release	Code review +3.1% per 25% more AI adoption (DORA)	Delivery stability falls as adoption rises (DORA 2024), and instability persists in 2025
Data migration and cutover	None of the studies above measures it	Mapping, reconciliation and sign-off by the business
Maintenance	Refactoring in nearly two-thirds of the time (McKinsey)	Every module you keep still has to be patched, tested and secured

The clearest large-scale result so far is a narrow one. AWS reports that Amazon moved tens of thousands of production applications from Java 8 or 11 to Java 17 with its own AI tooling, saving over 4,500 years of development work, and Andy Jassy put the typical upgrade at about 50 developer-days before and a few hours after. A version upgrade is a well-defined, repetitive transformation with a target fixed in advance. It is exactly the kind of work in which the decision has already been made and only execution remains.

Five questions to ask a vendor who prices AI savings into a proposal

Which phases does the saving apply to, and what share of the budget do those phases represent?
Is the work mostly new code or changes to existing code, and in which languages?
How was the saving measured: timed tasks, or developers’ own estimates?
What happens to review, testing and release effort, given what DORA found about delivery stability?
What will not be built? Ask for the list of modules, reports and integrations the proposal leaves out, and the evidence behind each exclusion.

In the arithmetic above, the fifth question moves the budget more than the other four together.

Code is getting cheap. Decisions aren’t.

That sentence sums up how we work at Pretius. AI tools make it cheaper to write a module; they do not tell you whether the module should exist. So before a modernisation or a new application is estimated, we look at three kinds of evidence: the code, how the system is actually used and the business process it supports. Then we decide, module by module, what to retire, keep, modernise or replace. When Sweco’s asset management platform was rebuilt, it was done module by module rather than as one big replacement.

If you are planning a build or a modernisation for 2027, we can go through the scope with you before anyone estimates the code. Book a discovery call.