What the Data Says About AI and Software Roles
A research-backed look at whether AI coding tools speed up or slow down software developers, why developers distrust AI-generated code, and what AI is doing to entry-level engineering jobs.
Short answer: The two most rigorous studies ever run point in opposite directions, and both are real. In a controlled greenfield task, developers with GitHub Copilot were 55.8% faster (Peng et al., 2022). In a randomized trial on mature codebases that developers already knew intimately, developers with Cursor and Claude were 19% slower (METR, 2025). The difference is not a contradiction — it is the difference between generating code and owning a codebase. Meanwhile entry-level engineering hiring has fallen sharply, but experienced engineering hiring has held up far better than most commentary claims.
This post grew out of an atomic note, “Impact of AI on Software Roles,” which was written from general impression. This version is built on named studies, surveys, and labor-market data, with sources at the end.
Key takeaways
- AI coding tools speed up greenfield work and can slow down mature-codebase work. 55.8% faster vs. 19% slower, both from randomized trials.
- Developers cannot perceive their own productivity change. METR participants predicted a 24% speedup, were actually 19% slower, and still believed afterward they’d been sped up by 20%.
- Trust is falling while adoption rises. Stack Overflow 2025: 84% use or plan to use AI tools, but 46% distrust output accuracy and only 3% highly trust it. Trust fell from ~40% to 29% in one year.
- “Almost right” is the expensive failure mode. 45% of developers named almost-but-not-quite-correct AI solutions as their top frustration; 66% spend more time fixing them.
- AI adoption correlates with worse delivery outcomes at the team level. DORA 2024: each 25% increase in AI adoption was associated with −1.5% delivery throughput and −7.2% delivery stability.
- The entry-level squeeze is real but mechanistically specific. Stanford Digital Economy Lab found a 16% relative employment decline for 22–25 year olds in AI-exposed occupations, concentrated where AI automates rather than augments.
- Experienced engineering hiring has not collapsed. SignalFire 2026: engineering hiring down only 11% at big tech while overall tech hiring is down 25%; software engineers rose from 46% to 55% of big-tech hires.
The study data at a glance
| Study | Sample | Setting | Result |
|---|---|---|---|
| Peng et al. 2022 | 95 developers | Greenfield HTTP server in JS | 55.8% faster with Copilot (CI 21–89%) |
| METR 2025 | 16 devs, 246 tasks | Own mature OSS repos | 19% slower with Cursor + Claude |
| DORA 2024 | 39,000+ professionals | Delivery outcomes, org level | −1.5% throughput, −7.2% stability per 25% AI adoption |
| Stack Overflow 2025 | 49,000+ devs, 177 countries | Attitude survey | 46% distrust vs. 33% trust; only 3% highly trust |
| Anthropic Economic Index | 500,000 interactions | Claude.ai + Claude Code | 79% automation on the agent vs. 49% in chat |
| Stanford DEL 2025 | ADP payroll microdata | Ages 22–25, AI-exposed jobs | −16% relative employment |
| SignalFire 2026 | LinkedIn-scale hiring data | Big tech + startups | New grad hiring −65%; engineering hiring only −11% |
Do AI coding tools make developers faster or slower?
Both. It depends on which kind of work you are measuring, and this is the single most misreported finding in the entire field.
Peng et al. (2022) randomized 95 professional developers into two groups to write an HTTP server in JavaScript. The Copilot group averaged 71 minutes; the control group averaged 161. That is a 55.8% reduction in completion time, p = 0.0017, 95% CI [21%, 89%]. Notably, developers with less programming experience benefited most.
METR (2025) ran the experiment most people actually care about. Sixteen experienced open-source developers worked on 246 real issues — bug fixes, features, and refactors — from repositories averaging 22,000+ stars and 1M+ lines of code, which they had been contributing to for roughly five years. Each issue was randomly assigned to allow or disallow AI assistance (primarily Cursor Pro with Claude 3.5/3.7).
The result: AI-allowed issues took 19% longer.
Why developers can’t tell which one they’re in
The most striking number in the METR paper isn’t the 19%. It’s the gap between reality and expectation:
- Beforehand, developers forecast a 24% speedup.
- After the study, they estimated they’d been sped up by 20%.
- Economics experts predicted 39% shorter; ML experts predicted 38%.
- Actual outcome: 19% slower.
Nobody in the entire study predicted a slowdown. The developers had already spent the study using the tools and still believed the opposite of what happened. Experts were wrong by more than 50 percentage points.
Why the two studies aren’t in conflict
Look at what each measured. Peng et al. gave participants a self-contained greenfield task with a clean repository, uniform instructions, and an unambiguous definition of done. There was no legacy, no conventions to violate, no maintainer whose taste had to be guessed.
METR gave participants work inside a system whose design had evolved for reasons written down nowhere. Only 39% of Cursor’s generations were accepted — code that “would have been good enough elsewhere” wasn’t good enough here, and rework followed.
There is a class of engineering work AI accelerates enormously (scaffolding, CRUD, getting a regex right, spinning up test fixtures) and a class it doesn’t (behavior that’s correct only within a system carrying undocumented context). The METR screen-recording decomposition is instructive: developers spent less time actively coding and searching, and more time reviewing AI output, prompting, and waiting on generations. The typing got faster. The whole thing got slower.
Why do developers distrust AI-generated code?
Because they can tell it fails in a specific, expensive way. Stack Overflow’s 2025 survey collected 49,000+ responses from 177 countries and documents an adoption-trust split widening in real time.
Adoption keeps climbing. 84% of developers use or plan to use AI tools, up from 76% the year before. But:
- 46% distrust AI output accuracy; only 33% trust it.
- Only 3% highly trust it.
- Trust fell from roughly 40% to 29% year over year.
- 75.3% say they don’t trust AI answers.
- 61.7% report ethical or security concerns about AI-written code.
- 61.3% want to fully understand their code.
The frustration data explains the rest. 45% named “AI solutions that are almost right, but not quite” as their single biggest annoyance, and 66% say they spend more time fixing almost-right AI-generated code than they would otherwise.
That is a rational response, not Luddism. Given plausible-looking output that is subtly wrong, and no cheap way to verify it without doing the work yourself, the rational move is to check all of it — trading transcription for verification, where verification is the expensive part.
Experience predicts skepticism. Experienced developers have the lowest “highly trust” rate at 2.6% and the highest “highly distrust” rate at 20%. The people with the most context to catch errors are the ones who most reliably expect errors. That’s uncomfortable when the buyer is the senior engineer who has to maintain what the tool produces.
But this isn’t a story about obsolescence. Asked what they’d want in a future with advanced AI, 75% said they’d still ask a person for help when they don’t trust AI’s answer. That’s a description of the job, not its disappearance.
Is AI adoption actually hurting software delivery?
At the level of teams and organizations, the evidence says yes, and it surprised the researchers.
DORA’s 2024 Accelerate State of DevOps Report, drawn from 39,000+ professionals, found that each 25% increase in AI adoption was associated with a 1.5% decrease in delivery throughput and a 7.2% decrease in delivery stability.
The same analysis found AI improved documentation quality (+7.5%), code quality (+3.4%), code review speed (+3.1%), and reduced code complexity (−1.8%). So the process measures went up while the outcome measures went down. DORA’s explanation is the plausible one: because code can be generated faster, teams produce larger batch sizes, which are slower to review and more likely to cause instability.
So individual and organizational effects genuinely point in opposite directions. That’s the honest summary, and it’s why “AI makes developers faster” is not settled.
What are developers actually delegating to AI?
Anthropic’s Economic Index analyzed 500,000 coding interactions across Claude.ai and Claude Code, and the chat-versus-agent split is the most useful finding.
- On Claude Code, 79% of conversations were classified as automation — the model performing the task directly.
- On Claude.ai, only 49% were.
- Directive conversations (hand over a task, minimal back-and-forth): 43.8% on Claude Code vs. 27.5% on Claude.ai.
- Feedback loops (model works, human pastes errors back): 35.8% vs. 21.3%.
Here’s the detail worth sitting with: compared to non-coding Claude use, coding showed an 18.3 point increase in feedback loops and an 11.2 point decrease in directive behavior. Coding is more automative than other work, yet within coding the dominant pattern is still a human supervising a loop and correcting it.
That is a meaningful narrowing of one task — typing — with very little elimination of the role around it.
The trend is also not monotonic. In August 2025, automation briefly overtook augmentation on Claude.ai (49% vs. 47%). By November 2025 augmentation was back in front, 52% to 45%, and coding’s share of Claude usage had fallen from a 40% peak in March 2025 to about 34%. Anyone forecasting an agentic future should note that delegation grew, then partly receded.
What is AI doing to entry-level software jobs?
Most of the evidence supports a real decline, concentrated at the bottom rung.
SignalFire 2025 found new grad hiring down roughly 50% from pre-pandemic levels. At Big Tech, new grads were 7% of hires, down 25% from 2023 and over 50% from 2019. Startups were under 6%. The New York Fed had the unemployment rate for new college grads up 30% since September 2022, against 18% for all workers.
Stanford’s Digital Economy Lab approached it with ADP payroll microdata in “Canaries in the Coal Mine”. After controlling for firm-level shocks, workers aged 22–25 in AI-exposed occupations saw a 16% relative employment decline, while experienced workers in the same occupations stayed flat. Employment for software developers aged 22–25 fell nearly 20% from its late-2022 peak; ages 35–49 grew.
Their mechanistically specific finding is the important one: entry-level employment declined where AI use was primarily automative, and showed muted change where it was primarily augmentative. That distinction separates “AI replaced the task” from “AI made the worker faster at it” — and only the former correlates with the decline.
Two caveats deserve weight. Most of the 2022–2025 contraction predates the point at which AI could plausibly do this work, so causation is inferential. And SignalFire’s own 2026 report substantially walks the story back: overall tech hiring sits at 75% of its pre-pandemic baseline, but engineering hiring is down only 11% at the Tech Majors versus 25% overall. Software engineers are now 55% of all big-tech hiring, up from 46% in 2019, and engineer attrition runs ~9% against ~13% for sales and design. New grad hiring is down much harder — around 65% at Tech Majors and ~76% at early-stage startups.
The pattern is better described as AI compressed the bottom rung and raised the value of the top than as AI ate software jobs.
One further distinction that BLS data blurs: computer programmers fell 27.5% from 2023 to 2025, while software developers fell 0.3%. Different occupations, very different trajectories.
What does this mean for the job itself?
The shape of it, as far as current evidence supports:
The typing got cheaper; the judgment got more expensive. Transcription-heavy work is faster. Anything requiring context the code doesn’t contain, or correctness you can’t take on faith, is not faster and may be slower.
The skill shift is real but mislocated. “Prompt engineering” as a career is mostly a mirage. The durable skills are the ones that make generated code cheap to verify — knowing what correct looks like, holding opinions about architecture, and reading a diff and knowing what’s wrong in thirty seconds. Stack Overflow’s data points the same way, with 61.3% wanting to fully understand their code.
There’s an unresolved structural problem. The junior role was never just labor cost; it’s how the industry manufactures seniors. SignalFire names the risk plainly: stop investing in early-career talent and the leadership pipeline empties out over a decade. Dario Amodei predicted in May 2025 that AI could wipe out roughly half of entry-level white-collar jobs within five years. The Stanford authors, far more cautious, still found the early-career effect real and concentrated where he said it would be.
The optimistic case rests on adjacent roles absorbing the pipeline — applied scientists, whose median salaries rose in the Stack Overflow data, and the emerging titles SignalFire predicts (AI governance lead, agentic AI engineer). Whether that happens in five years or twenty-five is genuinely unknown.
The constraint nobody has solved is verification. Everything productive here assumes a human who can catch a plausible wrong answer cheaply. That is the binding limit on this whole literature.
Frequently asked questions
Is AI making software developers faster or slower? Both, depending on context. Developers were 55.8% faster at a greenfield HTTP server with Copilot (Peng et al., 2022), and 19% slower on real issues in mature repositories they already knew well (METR, 2025).
Why did developers in the METR study think they were faster when they weren’t? Developers forecast a 24% speedup before the trial, were actually 19% slower, and still estimated a 20% speedup afterward. Perceived productivity and measured productivity moved in opposite directions.
What percentage of developers trust AI-generated code? In Stack Overflow’s 2025 survey, 46% distrust the accuracy of AI output and 33% trust it; only 3% highly trust it. Trust has fallen from about 40% to 29%.
Does AI adoption improve software delivery performance? Not according to delivery outcomes. DORA 2024 found a 25% increase in AI adoption associated with a 1.5% decrease in throughput and a 7.2% decrease in delivery stability, even while code quality and documentation quality improved.
Are entry-level software engineering jobs disappearing? Declining substantially, with nuance. New grad hiring at Big Tech is down over 50% from pre-pandemic levels (SignalFire 2025), and workers aged 22–25 in AI-exposed occupations saw a 16% relative employment decline (Stanford Digital Economy Lab, 2025). But experienced engineering hiring is comparatively stable and software engineers now make up a larger share of big-tech hiring.
Does AI automate or augment developer work? Both, and the split is shifting. Anthropic’s Economic Index found 79% automation on Claude Code versus 49% on Claude.ai. Across Claude.ai generally, augmentation led 52% to 45% in November 2025, after automation briefly overtook it in August.
Is prompt engineering a real career? The durable skill is verification, not prompt phrasing. Survey data consistently shows developers wanting to fully understand their code, and experienced developers distrusting AI output most.
Sources
- Peng, Kalliamvakou, Cihon & Demirer — The Impact of AI on Developer Productivity: Evidence from GitHub Copilot (arXiv:2302.06590, 2022/23)
- METR — Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (arXiv:2507.09089, July 2025)
- DORA / Google Cloud — Accelerate State of DevOps Report 2024 and Impact of Generative AI in Software Development
- Stack Overflow — 2025 Developer Survey, AI section and press release
- Anthropic — AI’s impact on software development and Economic Index: Economic primitives, January 2026
- Stanford Digital Economy Lab — Canaries in the Coal Mine (November 2025)
- SignalFire — State of Tech Talent 2025 and 2026
- IEEE Spectrum — How to Stay Ahead of AI as an Early-Career Engineer
- Ars Technica — Study finds AI tools made open source developers 19% slower