Coding Agents Shift Engineering’s Bottleneck From Code to Product Judgment

Laurie Voss’s central claim is that coding agents have made writing code nearly free relative to the time it used to take, but the rest of software development has not sped up at the same rate. The result is not simply faster engineering. It is a bottleneck that shifts: from writing code to reviewing it, then potentially to deploying, scaling, and securing it. The human work that remains, Voss argues, will be whatever agents cannot reliably take over.

He points to a mismatch between output and delivery. A slide in his talk, citing an NBER study of more than 100,000 GitHub developers, reports that developers using autonomous coding agents produced 741% more code but shipped 30% more releases. The extra output is real; the much smaller increase in releases suggests that work is piling up elsewhere.

For now, Voss says, that queue is code review. He cites research indicating that human review accuracy falls sharply beyond roughly 450 lines of code an hour. At that rate, a 10,000-line agent-generated diff would take about a week to review carefully. In practice, he says, developers are sending around diffs of that size without giving them that level of scrutiny. His slide also reports that agent-written pull requests wait 4.6 times longer for a reviewer, while time spent in review has risen 440%, incidents per pull request 240%, and merges without review 31%. The slide attributes those figures to LinearB, Faros, and Cisco and SmartBear, among other sources.

The underlying problem is a widening gap between producing plausible code and deciding whether to trust it. Review is the current constraint because humans cannot simply read more code faster. But Voss expects the constraint to move again if automated review improves.

AI reviewers are already taking on part of the work. Voss says GitHub Copilot was performing more than one in five code reviews on GitHub at the time of his talk, and that users could opt to let it approve a change before production. He also cites OpenAI’s stated practice that humans may review pull requests there but are not required to. These are examples he presented of how some organizations are using automated review, not evidence that the tools are reliable on their own.

In a study Voss describes, four AI reviewers examined the same 146 pull requests. Together they flagged 617 lines, but none of the reviewers flagged a line that all four agreed on. Each found issues; they did not converge on the same ones. Voss’s conclusion is not that these tools are useless, but that serious teams should not rely on a single reviewer.

Attempts to remove human review altogether have also failed, in his account. Voss cites Dex Horthy, who had spent six months urging developers not to read the code before saying he was wrong: skipping review had been a disaster, and the team had to replace whole systems. Voss says Amazon’s March 2026 postmortem linked outages to AI-assisted changes and that Amazon subsequently required senior engineers to sign off on those changes. He also points to an Anthropic postmortem describing bugs that got past multiple human and automated reviews, tests, and other checks.

Passing tests is not the same as producing a change a maintainer wants to merge. Voss’s slide summarizes a comparison in which agents pass 80% of a coding benchmark’s tests but pass only 54% when those same pull requests are judged by a maintainer. In his spoken explanation, he gives a related but distinct figure: Claude 3.5 passes a standard coding benchmark’s tests 80% of the time, while the Cognition benchmark asking whether a repository maintainer would merge a pull request falls from 87% to 54%. The 80% figure is about passing tests; the 87%-to-54% comparison is about performance on Cognition’s mergeability benchmark, not a second result from the same standard test.

The difference lies in what is being tested. Tests capture some requirements. A maintainer also weighs style, side effects, external systems, and the history of the codebase—knowledge that may live in the maintainer’s head rather than in the repository. That judgment is what Voss calls knowing what “good” means. Agents may identify bugs, but they do not yet reliably know whether a proposed change belongs in a particular system.

Voss uses the theory of constraints, associated with Eliyahu Goldratt’s The Goal, to explain why faster coding does not automatically produce faster software delivery. In any system, he says, there is a bottleneck; speeding up work that is not the bottleneck does little to improve the whole process. And once a constraint is relieved, another one becomes visible.

If code review becomes fast enough, Voss expects operations to be next: deploying a system and scaling it. Security may follow, including questions of privacy and preventing a system from going off the rails. He does not claim these are about to be solved. He expects meaningful progress on code review over the next year, but estimates that operations, scaling, and security could take five to ten years to automate substantially.

His longer-term argument also depends on a second assumption: that demand for software has no practical upper bound. Many websites and business systems are poor, he says, not because nobody knows how to improve them, but because making them good costs money and the people capable of doing that work are busy elsewhere. If agents make software production much cheaper, demand can expand to meet the new supply.

Voss sorts engineering work by how exposed it is to automation. Writing code has already become cheap. Reviewing and maintaining code are next. Shipping to production and scaling may follow. The work he considers more resistant is deciding what to build, defining what counts as good, and making the result delightful.

A request such as “keep track of my orders” is not a complete product specification. Many systems can record orders, but the right one depends on the business using it. A bakery, an auto-parts store, and a large manufacturing company may all need to track orders while needing very different workflows and definitions of success.

The missing information, Voss argues, is not necessarily in a model’s training data or weights. It is in the experience of the person running the business. Getting it out requires talking to that person and working out what the request means in practice. One of Voss’s slides puts it simply: “Only the baker knows what the bakery needs.”

That knowledge also resists reuse. A calendar app and a scheduling app may appear to address similar problems, but Voss says they can require different software and different definitions of success. Software that tries to serve every use case through configuration can still fail to fit any particular one; he invokes Jira as an example.

The constraint therefore moves from describing code to describing the problem: what needs to be built, and what would count as having solved it. Voss calls this essential complexity, drawing on Fred Brooks’s distinction between essential complexity—the difficulty inherent in the problem—and accidental complexity, such as the friction of tools, syntax, and typing. Brooks predicted that no single tool would make software development ten times more productive because the problem itself remains difficult. Voss sees the current gap between eight times more code and only 30% faster shipping as evidence for that distinction.

This is also why specifications matter. Voss quotes OpenAI’s Sean Grove: “Whoever writes the spec, be it a PM, a lawmaker, an engineer, a marketer, is now the programmer.” Code, in this view, is a lossy projection of the specification. The person who defines the intended result is doing work that agents cannot perform merely by generating more code.

Once a judgment can be written down and checked cheaply, however, it can become training material. Voss describes Cognition’s effort to define whether a change is “mergeable” and train against that benchmark. He also says Cursor’s Bugbot turns team members’ negative feedback into rules, with 44,000 learned rules across 110,000 repositories. The implication is that human judgment remains a bottleneck only while it stays tacit. Encode it in a spec, rubric, evaluation, or rule, and agents may learn to apply it—moving the constraint again.

Voss calls the role that combines customer understanding, product judgment, and delivery the product engineer. The work is to find out what people need, define what good looks like, and ship it, with agents doing much of the typing.

He argues that product-mindedness has shifted from a useful trait for senior engineers to a requirement for engineers more broadly. Voss cites Andrew Ng’s account of a team proposing twice as many product managers as engineers. Ng’s answer, as Voss describes it, has not been to hire more PMs but to blur the jobs. Engineers who understand users and make decisions about what to build, Voss says, can execute quickly.

The shift is visible in hiring. Voss’s slide reports that forward-deployed engineer postings rose 800% in nine months of 2025, with about 1,000 live postings across 462 companies. It lists average total compensation at roughly $240,000 and senior roles above $500,000. Voss says the same work appears under titles including solutions engineer, deployment engineer, applied AI engineer, and implementation engineer. The label is unsettled; the demand, he argues, is already visible.

He also points to a historical precedent. In the 1960s, systems analysts spoke with businesses to identify problems rather than primarily writing code. Over time, that function divided into product management and engineering. Voss’s prediction is that the separation will weaken again as engineers take on more customer discovery and product decisions.

He acknowledges the objection that this is not programming. In his formulation, it is not: the agent is doing the programming. But software development and programming are no longer the same job. Voss also cautions that his argument may reflect his own preference for figuring out problems. The claim that “the hard part” is the part one personally likes is a familiar trap in AI discussions, he says, and he invites listeners to judge whether he is making it too.

The shift creates a training problem. Traditionally, junior developers learned through well-specified tickets: implement a defined change, then receive feedback on what was wrong. Voss says agents now do that work. The ticket-writing and correction loop that once provided an entry point is being automated, even as organizations still need people capable of customer-facing product engineering.

Voss presented a chart of U.S. software developer employment by age, based on ADP payroll data and indexed to October 2022. The chart’s line for developers aged 22 to 25 had fallen 19% from its peak; employment among older developers had risen. His reading is that companies are not hiring juniors as they did before, while experienced engineers remain highly valuable. He calls the outcome a professional problem that needs attention, not a hypothetical future.

The scale of training available is far below the scale he thinks is needed. Voss’s slide says Google’s Associate Product Manager program produces about 50 people a year from roughly 12,000 applicants. He estimates that similar programs across companies produce around 1,000 people annually; he argues the industry needs at least 10,000. Universities teach data structures and bootcamps teach frameworks, he says, but few teach someone to spend a week with a baker and return with a useful specification.

His proposed alternative is deliberate apprenticeship in product work: put junior employees in front of customers early rather than leaving them in a ticket queue; have them write the specification and evaluation while an agent writes the code; and review their spec and customer-interview transcript, not only their code diff. Voss says IBM is already redesigning its entry-level role around talking to customers and writing specifications instead of programming.

The economic argument is that developing product engineers may be cheaper than buying experienced forward-deployed talent at senior-engineer compensation. But Voss’s urgency is also about people already entering the field. The ladder has changed before the profession has built a new way to climb it.

Voss does not treat the transition as an uncomplicated improvement. Many people entered programming because they love its craft: clean abstractions, difficult bugs, and making a machine do exactly what they intended. They did not sign up to interview business owners about their needs, and he expects many to dislike the new role.

He offers no consolation that the old work will remain a highly paid occupation at the same scale. Programming as craft, he says, will survive much as woodworking survived factory furniture: people will continue to do it because they love it, but it may not be what employers pay hundreds of thousands of dollars a year to have done. He suggests that some programmers may have valued not only typing but the moment before it, when the problem becomes clear and a repeatable system takes shape. That creative work remains.

His forecast is ten difficult years. The tools are arriving before the profession has adapted; the junior ladder is shrinking before new training systems exist; and people who love programming will have to decide whether they also want the work replacing it. Voss nevertheless expects the profession eventually to grow, as it did after compilers, frameworks, and open source reduced the need for earlier forms of manual work. In his prediction, software development will mean something different and become a larger profession.

For teams acting now, Voss recommends reserving human review for code with a large blast radius: public APIs, authentication, schemas, and other areas where mistakes carry significant consequences. Delegate work that is cheap to verify, and build automatic checks that can tell an agent whether its output meets the standard. Put experienced engineers’ judgment into specifications, rubrics, and evaluations rather than relying on knowledge held only in individual heads.

He also argues for measuring delivery rather than production. Lines of code and pull requests opened are not the goal. Teams should track what ships, whether reviewers’ comments lead to changes, and the gap between passing tests and a maintainer deciding a change is mergeable. Production is the last reviewer; observing what a system does there and evaluating it is part of the feedback loop.