About this video
AI coding agents raised code output 30% across 718 companies. The finished software tells a different story.
Where do you think the extra code went?
We read the new Harvard study by Fiona Chen and James Stratton page by page: 300 million real work events from Jellyfish's engineering data, what changed when companies switched on AI assistants (GitHub Copilot, Cursor) and agents (Claude Code, Cursor's agent), why shipped work didn't measurably rise, how code review became the bottleneck (49% longer reviews, nearly twice as many pull requests sent back), why AI reviewers haven't fixed it yet, what happened to jobs, the paper's kitchen-style model of speed vs quality, what it does not claim, and what to do about it if you write code, lead a team or build AI tools.
Chapters
0:00 Thirty percent more code
0:33 How software gets made, and how they measured it
1:42 Finding 1: More code
2:27 Finding 2: Nothing more shipped
3:17 Finding 3: The traffic jam is code review
4:30 Finding 4: AI reviewers haven't fixed it
5:06 Finding 5: Same jobs, more reviewing
5:57 Why: the kitchen model
6:52 What the paper does not claim
7:32 What to do about it
8:37 Recap
Paper: Artificial Intelligence in the Firm: Bottlenecks in Software Production. Fiona Chen, James Stratton (Harvard University). Job market paper, version of August 4, 2026 (first version January 7, 2026). https://fion.ac/jellyfish.pdf
Short excerpts of the paper (its title, Figure 1 and interview quotes) are shown with attribution for commentary and review. The charts, pipeline, kitchen and characters are our own illustrations redrawn from the paper's published numbers.
Coverage: Ars Technica, October 2026: https://arstechnica.com/ai/2026/10/ai-coding-agents-generate-more-code-but-not-more-software/
Music: Voice Over Under by Kevin MacLeod, CC BY 3.0, via Wikimedia Commons (https://commons.wikimedia.org/wiki/File:Voice_Over_Under_(ISRC_USUAN1600001).mp3)
Pranav Innovates: AI research, explained. Subscribe to stay up to date.
What the video says
Companies gave their engineers AI coding agents, and the code they wrote jumped thirty percent. So, thirty percent more finished software. Right? It didn't measurably move. One engineer said the only person allowed to approve their code was so swamped, he could review it just once a week. Three hundred million real work events. Seven hundred and eighteen companies. By the end, you'll know where the extra code goes, and the one question to ask anyone who says AI made their team ten times faster. Let's read it, page by page. First, the four steps every piece of software goes through. Plan, code, review, ship. Plan: the work is written down as tickets. Code: an engineer writes it and bundles it into a pull request. Review: a teammate reads it, comments, and approves it or sends it back. Ship: the code goes live, and the ticket is marked done. The data comes from Jellyfish, a company that tracks engineering work. Seven hundred and eighteen of its client companies consented to their data being used for research. That's more than seven hundred thousand workers, from twenty twenty one to March twenty twenty six. And they can spot AI from the fingerprints agents leave behind. Bot accounts, and notes in the code history like, co-authored by Claude Code. Companies switched AI on at different times, so the researchers compare the ones that just switched it on with the ones that haven't yet, month by month. It's as close to an experiment as real life gets. Two kinds of tools. Assistants, like GitHub Copilot, autocomplete as you type. Agents, like Claude Code or Cursor's agent, take a whole task and do it. The agents are the story. Finding one. Agents really do make people write more code. Per engineer, per month: lines of code, up thirty percent. Commits, up twenty. Pull requests, up twenty three. And it's not just wordy AI padding out the lines. Each pull request stayed the same size. There were simply more of them. Assistants did far less. Commits rose nine percent, and the rest was too small to be sure of. One senior engineer said that for simple tasks, Claude Code writes a draft in two seconds, and it's right ninety percent of the time. A speed-up on the order of a hundred times. For builders, that's the baseline. Agents do raise output per engineer. The real question is where that output goes next. Finding two. Quick guess before I show you. With thirty percent more code, how much more finished work got shipped? Twenty percent? Ten? Statistically, zero. The tiny rise they measured could just be noise, and the data rules out anything bigger than twelve percent. Whole features, the bigger projects made of many tickets? Same story. Flat. Maybe the tickets just got bigger and harder? They checked. They had an AI rate a sample of tickets by how long each should take, then trained a model to rate all sixty million plus. The average size didn't change. For builders: if your dashboard shows commits and lines of code going up, that is not proof anything shipped. The paper's own advice is to measure the whole process, not the middle of it. Thirty percent more code. No measurable rise in shipped software. So where did it go? Finding three, the heart of the paper. The extra code piles up at review. First, a surprise. Engineers don't spend most of their day writing code. The work data says about forty two percent. A survey of a hundred engineers says about thirty. Speed up the coding, and the rest of the day is still there. After agents arrive, the time from opening a pull request to merging it rises forty nine percent. From about seven days, to about ten and a half. The share sent back for changes nearly doubles, from about one in eight to one in four. By month twelve, it's about one in three. And each review gets thirty five percent more comments. Remember that engineer? Only one senior person could approve the code, and he could review it once a week. Then outages were caused by AI-written code, so a new rule arrived. AI code needs senior approval. The jam got worse. And the data says pull requests didn't get any bigger, but they feel harder. One senior engineer said it's honestly becoming more anxiety inducing to even open one, and begin the review. For builders: a pull request at these companies now waits about ten and a half days. That wait, not the typing, is your real cycle time. Finding four. The obvious fix: let AI review the code too. Nearly eighty percent of these companies have tried an AI review tool. But only about twenty three percent of review comments come from AI, and just one pull request in nine gets an AI comment at all. One senior engineer put it simply. The AI tools lack the comprehension of the overview of the entire feature, and what it is supposed to do. Humans are still the checkpoint. For builders, that's the gap. An AI that leaves comments isn't enough. What's missing is one that understands what the whole feature is supposed to do. Finding five, the one everybody asks about. Did these companies need fewer people? No. Total headcount didn't change, and the data rules out a drop of more than about three percent. For engineers alone it's noisier, but it still rules out cuts bigger than about fourteen percent. And juniors and seniors, no significant change either. But the work changed. The share of people doing code review rose fourteen percent, while the share writing code stayed the same. Nobody stopped coding. More of the same people started reviewing too. Two cautions from the authors. This is the short run, and jobs can react slowly. And if AI users win customers from rivals, the losses could show up at the rivals instead. For builders: when you roll out agents, you're also deciding who reviews. Make it someone's job, before the queue makes it everyone's problem. So why does this happen? The paper builds a simple model. Picture a restaurant kitchen. Cooks make the dishes. A head chef tastes every plate, and sends back the bad ones before they reach a customer. Give the cooks AI, and two things can change. They get faster, so more plates arrive. And some plates come out worse, so each one needs a closer taste. Faster cooks alone just need more tasters. Each plate takes the same time to check. But worse plates make every single check take longer. The data shows both. More pull requests is the speed. More send-backs, and more comments on each one, is the quality. And the fixes are different. If it's speed, move people into review. If it's quality, no reshuffle fixes it. The AI's work has to get better. Either way, the model says the share of people doing review goes up, never down. Which is exactly what finding five found. Now, what this paper does not show. It hasn't been peer reviewed yet. It's a job market paper. And the data comes from one analytics company, from clients who opted in. It measures companies switching AI on, not individuals. A year in, only about one in five engineers at a typical company was using agents. So the effect per user is probably bigger. Finished tickets stand in for output. They don't capture quality or revenue. And the data ends in March twenty twenty six. Agents have improved since. So the lesson isn't, AI coding doesn't work. It's this: the gains are real, and right now they're stuck in one place. So what do you do with this? If you write code, the valuable skill is shifting from writing to judging. One junior engineer said their work became less about writing code directly, and more about reviewing and correcting what Claude produced. If you lead a team, the paper's lesson is simple. Measure what ships, not what's typed. And plan review capacity the day you buy agents, because the bottleneck moves. Now, my suggestions, not tested in the paper. Have agents hand over smaller changes, with tests, and a plain English note on what changed and why. Let AI take the first pass, so human reviewers spend their time on the design. And if you build AI tools: everyone built the faster cook. The gains are stuck at the head chef. Tools that make code easier to test, trust and review are where they get unlocked. My take. I use coding agents every day, and this matches what I see. Writing the code is no longer the hard part. Knowing it's right is. The teams that win won't be the ones that generate the most code. They'll be the ones that can trust their code the fastest. Recap. Agents: thirty percent more code. Shipped software: no measurable rise. Jobs: the same number, with more people reviewing. Reviews: forty nine percent slower, with nearly twice as many sent back. AI reviewers: not a fix, yet. Speed up one step, and the bottleneck just moves. So the next time someone says AI made their team ten times faster, ask them one question. Faster at what? The paper is linked below. If you want the next big paper explained like this, subscribe. See you in the next one.