Devin’s 80% Commit Share Shows Background Agents Becoming Production Infrastructure

Cole Murray describes the current turn in software agents as a move away from hand-held, foreground interaction toward cloud agents that can take a specification and return a completed pull request. In his account, the important inflection was around December 2025, when models such as Opus 4.5 and GPT 5.2 became capable enough that, given a sufficiently good spec, the agent could drive much of the work autonomously.

That changed the interaction model. Instead of sitting with a model inside an IDE and correcting it step by step, teams could begin treating the agent as a background worker: give it a task, let it run in a cloud environment, and inspect the result when it is ready.

Walden Yan says Cognition saw the same shift internally, but as an acceleration of a curve already underway. Sonnet 3.7 had been a major jump for Devin because Cognition could strip out parts of its system that were no longer needed once the model became more intelligent. More recent models, he says, pushed autonomy far enough that teams which once debated whether they needed to stay “in the weeds” with the model in an IDE now seriously ask whether the work can move fully into the cloud.

The operating evidence inside Cognition is not presented as a benchmark score but as production behavior. Yan says Devin’s merged PR usage grew 7x over roughly two or three months while engineering headcount rose by about 10%. A chart titled “Combined Devin Commit % Over Time” showed non-merge commits attributed to Devin across three Devin repos rising from 16% in January to 80% by mid-March.

Yan frames that as the practical reason “everyone is building their own Devins.” Once background agents are visibly producing merged code inside real repositories, they stop looking like demos and start looking like internal infrastructure.

Murray’s OpenInspect came out of the same observation, but from a different angle. His clients were using tools like Claude Web and OpenAI’s Codex and running into workflow friction. One concrete problem: Slack-launched sessions were tied to the person who invoked them. If a PM started the session and then wanted engineering to continue from there, engineering could not simply “jump in” to the same session. The result was copy-pasted fragments or a single returned response rather than a shared, inspectable workstream.

OpenInspect, which Murray released as an open-source implementation inspired by Ramp’s background-agent blog post, is meant to provide a foundation companies can fork and customize. Its GitHub repository describes a headless background coding agent that can run in cloud development environments, work through web UI, Slack, GitHub PRs, Linear tickets, or local CLI, run scheduled tasks, spin up ephemeral Docker instances, and use Playwright for visual UI tests. The README presents long-term memory as part of the project’s ambitions or capabilities, while Murray later says memory is not solved in OpenInspect and that there is an open issue for it. The README also emphasizes a single-tenant security model: the system executes arbitrary LLM-generated code and is designed for trusted organizational deployment.

Murray says he did not try to turn OpenInspect immediately into a venture-backed seat-based product because, in his view, background-agent systems are likely to become critical company infrastructure. He wanted teams to be able to fork the system and add whatever customizations their environment required. He also argues that competing for “twenty dollars a seat” is a difficult business when much of the stack is owned elsewhere: sandbox providers such as Daytona and E2B, model providers, and integration layers all capture parts of the value.

Yan answers the monetization question differently for Cognition. Cognition sells not just the agent product but the infrastructure required to run it and the organizational work required to adopt it. Early Devin infrastructure had to be built because raw cloud VMs were not designed for repeated sleep-and-wake agent workloads; booting and restoring machines could leave the agent “out cold for like ten minutes.” Cognition now tells customers it can make the compute work, including in the customer’s cloud if needed. Yan also says a major part of the work with large enterprises is onboarding: setting up integrations, automations, workflows, and literacy so the company can actually use AI agents in day-to-day engineering.

Cole Murray lays out the first major design choice in a background-agent system as “harness in the box” versus “out of the box.” In the first model, the agent itself runs inside the sandbox where code is being edited and tests are being run. In the second, the agent’s “brain” runs in a worker or control plane outside the sandbox, while the sandbox acts as the hands: the environment that receives tool calls and carries out actions.

The in-box design is simpler because the state is localized. The agent, its process, its files, and much of its execution state live in the same environment. That reduces orchestration complexity. But Murray identifies the major tradeoff as security: if the agent runs in the box, secrets may need to be present in the same environment. Because AI systems can behave unpredictably, teams can accidentally exfiltrate secrets or create other unintended behavior unless they design carefully around that risk.

The out-of-box design is more complex because state has to be managed across the control plane and sandbox. But Murray says it is, in his opinion, the better architecture.

Walden Yan says Devin was built from the start around a similar principle Cognition calls separating “the brain from the machine.” The separation lets customers reuse existing dev-box infrastructure without having to turn those boxes into places that contain the dependencies and secrets required by the agent’s brain. It also creates a cleaner permission boundary.

His GitHub example shows the issue. A company may have a GitHub app, and multiple users with different permissions may interact with the agent through it. If all users share the same GitHub application path and there is no separation between the system deciding what to do and the machine holding credentials, enforcing user-level separation becomes difficult. In Devin’s model, Yan says, whatever is placed on the machine scopes what the agent can do. The instruction is to put only the most scoped secrets on that machine. The brain is not accessible from the machine, so a user who has freedom to operate in the machine cannot tamper with the more secure parts of the agent.

The same architectural vocabulary appears in platform materials referenced during the discussion. OpenAI documentation described “a more capable harness for the agent loop,” where an Agents SDK handles the loop, tools, and durable execution. Anthropic’s “Scaling Managed Agents: Decoupling the brain from the hands” separated tools, session, harness, sandbox, and orchestration. Murray identifies that Anthropic-style pattern as out-of-box: more work, but a better architecture in his view.

OpenInspect today uses a control-plane/data-plane design. Its architecture diagram places the control plane on Cloudflare, with durable objects, a SQLite database, WebSocket hub, event stream, and GitHub integration, connected to a D1 database for repo-scoped secrets and a data plane on Modal containing the session sandbox and full dev environment. Murray says OpenInspect currently supports Modal sandboxes; a contributor added Daytona; E2B is on the roadmap; and the abstraction is designed so contributors can add providers.

Murray also says OpenInspect may eventually move more fully out of the box. Some newer tools already call back into the control plane so secrets do not need to live inside the sandbox. Long term, he expects to pull the actual agent out of the box, though for now he says the current design is acceptable.

The unresolved operating problem is not only how smart the model is, but what is actually on the machine where the agent works. Walden Yan says one thing he has not seen many other players do well is manage the agent’s working environment: the repository changes, dependencies change, credentials are needed, services must be started, and the environment has to remain usable. Internally at Cognition, he says, this is called “repo setup,” and it has been a perennial hard problem since the start of the company.

Cole Murray sees the same issue in consulting. Many teams do not have strong developer-environment setup even for humans. Sometimes the process is effectively “go talk to Bob and get the secrets,” which cannot work when an agent needs to configure itself. Many clients use Docker Compose or microservices for local development, and Murray argues that whatever makes a good developer setup for humans tends to make a good sandbox setup for agents.

Docker is useful, but the speakers distinguish between Docker as application infrastructure and Docker as the agent boundary. Yan says Docker containers are not a true security boundary and also become awkward when real applications themselves need Docker. If an agent’s sandbox is a Docker container and the application under test uses Docker, teams run into Docker-in-Docker complexity. Cognition chose full VMs because many realistic dev environments require Docker inside the dev machine, and because full isolation is important for untrusted user workloads.

That VM choice creates its own complexity. Yan says Cognition is working on support for machine-specific development, including macOS and Windows use cases. He mentions .NET applications that need Windows and iOS or macOS development that needs Apple environments. Cognition’s separation of brain and machine supports this architecture in principle, though he says work is still in progress. Android development support is already in beta and required nested virtualization: the Devin machine is itself a virtualized Firecracker instance, and the Android emulator runs inside it. That creates performance issues, but unlocks Android work.

The infrastructure details matter because agents are not just editing files; they need to wake, sleep, resume, test, and sometimes roll back quickly. Yan describes an earlier Cognition effort called Blockdiff, a file format for VM disk snapshots. Cognition’s Blockdiff material says Devin writes and runs code in a VM environment, and explains that VMs were chosen both for isolation and because realistic dev environments often require Docker. The displayed comparison put EC2 startup time at 45–60 seconds versus 2–4 seconds for Cognition’s “otterlink,” and EC2 snapshot time at 30–60 minutes versus 5–10 seconds for otterlink.

The intuition, Yan says, is that if a VM has a terabyte of disk but the agent only wrote 100 lines of code, saving and restoring should not require work proportional to the whole terabyte. Cognition built a system where saving and bringing back the machine is proportional to the diff in the file system. The Blockdiff material further explains why very incremental VM snapshots are useful: a base operating system, dev environment, session state, and rollback checkpoints can be layered rather than repeatedly saved as full disk images.

Yan also gives a separate example of the kind of systems knowledge that can change the product. Some teams building coding agents ran into slow grep on agent machines and responded by building custom grep indexes. Cognition’s infra team found a lower-level cause: some virtual machines used network file systems backed by remote storage, so grepping could involve network calls. The solution was not a new AI-generated index but changing the underlying file system setup.

For Yan, this is part of what it means to sell an agent product. It is not “technically agents,” as Shawn Wang puts it, but it is agent infrastructure. Yan says the advantage of owning both the agent and the infra is that Cognition can figure out deployments in environments such as VPC, on-prem, or Fed GovCloud without waiting for an underlying infra provider to support them.

The testing problem is easy to understate if it is framed as browser control. Walden Yan pushes back on the idea that testing an application with an AI agent is mainly a “computer use” problem. Computer use, in his definition, is the literal UI-control question: if there is a button to click, can the model emit the right coordinates to click it? That part has improved with recent models. But he says the real testing problem is much harder.

For an arbitrary change spanning frontend, backend, and perhaps deeper services, the agent has to reason through how to run the relevant applications together with the right version of the code. Then it has to determine how to trigger the changed behavior. That may require logging in as an admin, turning on a feature flag, running two sessions, or sending a specific word into one of them to cause a particular event. Yan says that requires codebase context and orchestration, and in some cases Cognition has found that no single frontier model can solve the whole end-to-end testing task alone. They have had to orchestrate different frontier models together.

Cole Murray agrees that computer use is a subset of the broader testing problem. A background agent has one advantage: it has the codebase locally and can inspect what changed. But knowing how to test the change is codebase-specific. It is not solved out of the box by generic browser-control ability.

Screenshots and video become important because the human merge decision depends on trust in the result. Murray says OpenInspect added screenshot support, with video support possible via plugin, after customers heavily requested it. OpenInspect documentation describes screenshots captured through Playwright and stored as artifacts.

The Devin example in Slack makes the value concrete. After a PR is complete, a user clicks “Testing approved,” and Devin begins visual testing on a Vercel preview. It returns an embedded video, with captions describing what it is checking: a card should show “@ AI Hero,” the expanded view should show the company “AI Hero” with social links, and a speaker page should show a new LinkedIn link. Shawn Wang says the annotations matter because otherwise the reviewer does not know what they are looking at or what the agent is trying to demonstrate. Yan says adding bottom annotations made a large difference.

The point is not just visual polish. Yan says there is a long tail of small details that affects the end metric: how fast a human actually merges the code. If a video tells the reviewer what was tested and shows the cursor carrying out the actions, Wang says he can reach a point where he “doesn’t need to see the code” for that change because he knows it works.

Yan extends the same product-design point to GitHub. Devin does not only create PRs; users can interact with it directly on GitHub. Comments on the PR are received by Devin and addressed there. With Devin Review, an AI reviewer may post comments on Devin’s own PR, and Devin may respond to its own review comments. That creates obvious risks of loops, but Yan says Cognition works to prevent infinite loops by making comments high signal and making the agent thoughtful about which comments to fix and which to reject. One of his favorite moments, he says, is when Devin tells him he is wrong.

Murray sees AI review as a critical part of the background-agent system. OpenInspect includes a GitHub code reviewer with a controllable prompt. It can be tagged on GitHub to resolve merge conflicts or address requested fixes, though it does not automatically comment yet. The capability exists, but the automation has not been fully switched on.

The most common need beyond standing up a background-agent system is integrating it into the company, Cole Murray says. A system that can launch sessions is useful, but the bigger leap comes when it connects to production logs, read-only production database credentials, Confluence or internal knowledge bases, GitHub, Slack, ticketing systems, and other operational surfaces. In compliance-heavy environments, access control becomes a major design problem.

Walden Yan says Cognition has found that MCPs are not enough for first-class integrations. MCP exploded because it made it easy to connect agents to tools, but good product experience often requires building custom paths. Slack is his main example. Giving an agent a Slack MCP may let it post messages, but Cognition built Devin as a coworker in Slack from the ground up. That requires webhooks coming back, natural responses, thread behavior that does not spam the company, and tuning around long back-and-forth interactions.

Yan says he would like something “more expressive than MCP”: not just a set of tools, but a proper bidirectional system that allows the agent to have the right experience across interfaces. When the MCP spec becomes too complicated, he says, it begins to lose its original promise of simple one-step connection and starts to resemble first-party integration work anyway. Murray adds a pragmatic criterion: if an integration is critical and nearly every session uses it, a company probably should own it and optimize it rather than rely on whatever is off the shelf.

This integration question also shapes agent use cases. Murray identifies SRE as the easiest and most common cloud-agent use case across his clients. Alerts may come through Slack, Datadog, Sentry, or elsewhere; the desired workflow is for the agent to be first responder. It does not necessarily need to fix the issue immediately. The value is that it can collect context from logs, databases, and playbooks before a human arrives. In some cases, the agent can produce a PR directly from the incident. OpenInspect supports Sentry and generic webhooks, and Murray says Datadog could be added by a contributor.

Yan calls this “auto triage.” Internally, Cognition wants every alert or bug report to trigger Devin to begin triaging before anyone else does. They have leaned into making the workflow possible without leaving Slack: the report comes in, Devin responds, and people can ask follow-up questions in the same thread with full codebase context.

Murray also sees non-engineering use cases changing who contributes code. PMs with quick bug fixes may no longer file an issue for engineering; they prompt through Slack and receive a pull request. Customer support can tag an issue in Slack when a customer hits unexpected behavior; the agent gathers context, and engineering can enter with the investigation already assembled. Yan adds support and sales as beneficiaries of a chat interface that assumes no technical knowledge but has access to the codebase.

Memory is not solved yet, according to Cole Murray. Across clients, he sees teams using skills or updating Claude.md as partial solutions, but he considers memory as a whole a difficult retrieval problem. It is not only about retrieving stored facts; it is also about generating the right memories in the first place. The system should not remember overly specific one-off instructions as permanent policy.

Walden Yan says Devin’s first durable memory system was called Knowledge. The premise was that users should not have to proactively write extensive documentation for Devin. Instead, when a user corrects Devin — “that’s not quite the way you’re supposed to use git” — Devin can ask whether it should remember the instruction for the future. The user can approve or reject quickly, and the memory base builds over time.

Yan says that roughly 95% of Devin memories, or “some crazy stat like that,” came from auto-generated memories rather than user-authored documentation. That confirms the product intuition but creates hard tuning problems. Generation must avoid overgeneralization: if a user once asks Devin to open a draft PR, that should not become “everyone forever should get draft PRs.” But it may be valid to learn that a particular person generally likes draft PRs. Retrieval has the opposite problem: if thousands of memories exist, the system must retrieve the useful ones without filling context with irrelevant information. Yan says this requires a surprising amount of eval work to keep memory reliable as models change.

Memory pruning is also unresolved. Yan says today Devin can edit memories. If a memory says Cole likes draft PRs and Cole later says not to do that, Devin can ask whether to update the memory. But Yan does not present this as the final architecture. Because agents have become good at using file-system-like structures natively, Cognition is exploring whether memories should feel more like a file system the agent can navigate.

Other memory patterns provide useful contrasts, but they appeared in referenced materials rather than as claims the guests developed at length. OpenHands documentation describes memory as plain Markdown files in the agent workspace. A Latent Space visual about Claude Code’s memory described a three-layer design: a MEMORY.md index to other knowledge, topic files loaded on demand, and full session transcripts that can be searched, with an “autoDream” mode for sleep-time merging, deduping, pruning, and contradiction removal. Yan separately says Cognition’s team was surprisingly not very interested in the Claude Code leak and did not spend much time on it, but the file-system direction remains part of his own exploration.

Yan also describes a more continuous use of memory: an agent as a permanent PM for a product area. In that mode, a Devin might sit in a Slack channel for a product such as DeepWiki, maintain its own memory document, track priorities, know who owns upcoming work, tag people on a recurring basis, and create tickets that humans or other Devins later execute. The agent moves upstream from writing code to maintaining product-area state.

Shawn Wang describes a related automation: an agent that researches competitors weekly and suggests ideas, with rejected suggestion patterns fed back into the prompt. His feature request is that rejecting a PR should update memory automatically so the scheduled automation improves without manual prompt edits. Yan replies that Cognition may change the system soon.

Walden Yan is known for “Don’t Build Multi-Agents,” a Cognition post about context engineering whose displayed principles include “Share context” and “Actions carry implicit decisions.” He now says that post may eventually need revision. Multi-agent systems were “very much not at all possible” a year earlier, in his view, but there are more serious experiments now.

Still, he distinguishes between true multi-agent collaboration and tool-call-like delegation. When a main agent spawns a sub-agent to find a file or implementation, the benefits are often context-management benefits: the sub-agent spends tokens and tool calls, then collapses its work into an answer for the main agent. Devin does similar work when it calls DeepResearch and receives results. Yan says that feels like a tool call rather than two collaborators exchanging views.

The practical architecture remains manager-to-sub-agent. Yan says Cognition has given Devin an MCP through which it can message or create other Devins, but arbitrary inter-agent messaging creates a chaotic world. Day-to-day usefulness still mostly comes from one top-level Devin decomposing the work and assigning isolated pieces to other Devins, each with its own box and little room for conflict.

Cole Murray says OpenInspect can launch sub-sessions, create other environments, and monitor them. If the harness is out of the box, those sub-sessions are additional sessions running in the worker/control plane. The main design question is how the top-level agent interacts with them. He sees extra complexity but not a fundamental blocker if the architecture is already solved.

The more immediate risk is not that teams lack enough agents. It is that they let agents merge too much unreviewed work. Yan says Cognition has run experiments trying to build entire real products through pure vibe coding on top of itself: auto-merge, no code review, no human inspection. The benchmark became how many weeks the codebase could survive before it had to be trashed and rewritten. In December’s state of the art, he says, the answer was about two weeks. After that, even changing a button color became painful because the button might exist in ten different slightly varied implementations, with one missed or inconsistent.

Murray states the broader warning sharply: the idea that engineers do not need to look at code is generally bad. In teams embracing AI-native coding, he says, “your codebase regresses to your worst engineer.” If an enthusiastic engineer does not audit AI-generated code, their bad patterns enter the codebase. The AI then treats those patterns as precedent and amplifies them. A 20-branch if-else pattern becomes the learned way to implement similar features. Duplicate helpers proliferate. Scheduled cleanup, by humans or tools that detect duplication, becomes necessary.

Shawn Wang suggests that stricter module boundaries may be one way to keep this manageable: architects should define hard contracts between modules, allow freedom inside the box, and require human signoff for changes across boundaries. Yan agrees more generally that humans remain valuable in deep infrastructure and architectural judgment, especially where the correct solution depends on expertise that the AI may route around with superficially plausible code.

AI-written code is beginning to have recognizable failure patterns, and Cole Murray and Walden Yan both describe teams turning those patterns into lint rules, Semgrep checks, review systems, and conventions.

In Python, Yan says AI-written code often uses patterns he dislikes, such as checking whether objects have attributes or using getattr defensively when the code should simply error if the attribute is absent. Murray interprets this as reward hacking: the model tries to avoid failure at all costs, even when direct code would be clearer and safer. For some clients moving toward autonomous coding, he has added lint rules so that use of getattr fails the PR.

Murray sees a similar “backwards compatibility at all costs” pattern in GPT models, and increasingly in Claude 4.6. The model creates odd import/export compatibility layers rather than renaming modules cleanly, because it does not want failures. He says this behavior is easy to detect with Semgrep or similar tools once a team knows what to look for. Untyped tuples and overly broad types such as dict[str, any] are another recurring problem that linting can catch.

Yan adds that Opus 4.7 writes many comments — not line-by-line noise, but paragraph-like PRDs above functions. He says the comments are not pure slop; they often include reasoning, alternatives considered, and why those alternatives were not chosen. But they are still too verbose for his taste. He can imagine this direction being useful if systems need to maintain themselves over the long run, but he would like control over verbosity.

That leads to git-AI, a GitHub project whose README says it tracks AI-generated code in repositories by linking AI-written lines to the agent, model, and transcripts that generated them. Yan describes the idea as storing the prompts sent to the agent alongside the code in git metadata, so future agents and review bots can recover the intent and architecture decisions behind the code. In his view, this may be a cleaner version of the model’s instinct to put large rationales into code comments: the context lives beside the code, but not necessarily inside every function.

The deeper recommendation is to make codebases agent-ready. Yan says teams should make it possible for agents to do the full loop: write code, run it, and test it locally without privileged production access. That often means local database setup, Docker Compose, local Postgres, and test paths that do not require live services. Cognition has shifted core components toward purely local development for this reason.

Older companies face more migration work because large codebases and microservice architectures often grew without local-first assumptions. Some companies began before Docker existed. Shawn Wang notes that Devin is good at making mock servers, and suggests that observing application traffic could help reconstruct local mocks. Yan calls that interesting, but the main point remains narrower: the more local and reproducible the system is, the more useful the agent becomes.

Windsurf 2.0 is Cognition’s attempt to connect local and cloud agent workflows. The Windsurf page referenced during the discussion describes “Agent Command Center and Devin in Windsurf.” Shawn Wang frames the problem this way: if users rely on background agents enough, sometimes they need to bring one into the foreground. The handoff from local to cloud is hard.

Walden Yan says the main gap is still testing speed. When Devin can test by itself and send back a video, the experience is “freaking magical.” But some cases remain difficult enough that a human needs to pull the work down locally. Windsurf is meant to be a command center for local and background agents: review something locally, move other agents to the background, approve a background fix, continue to the next item, and ideally not leave the window.

The local and cloud agents should not behave exactly the same, according to Yan. Local agents should be faster and let the user make calls. They should not necessarily try to autonomously run full testing before responding. Background agents, by contrast, should assume that the next message to the user must contain everything the user needs; they should keep running rather than stop prematurely, and they should return with testing completed where possible.

That distinction becomes more important as usage expands beyond engineers and as costs rise. Cole Murray says common budget numbers he hears are roughly $1,000 to $5,000 per engineer. He has not heard $50,000 per engineer as a normal frame of reference. Yan says he has seen numbers go that high.

Yan expects the coming year to feature both very expensive, very smart frontier models and a recognition that many tasks do not require the frontier. Sub-frontier models may be good enough for a large share of work. He is interested in hybrid systems that use sub-frontier components for speed and efficiency, then call out to frontier components where frontier performance is necessary.

That architecture echoes the rest of the production stack. A background-agent system is not a single model call. It is a control plane, sandbox, machine state, repo setup, scoped secrets, integrations, memory, review, testing, and sometimes multiple models or agents working together. The economics will likely be managed at that system level rather than by assuming every step deserves the most expensive model.

Yan describes the broader direction as companies wanting to turn themselves into “autonomous coding factories.” But the details make that less like a slogan and more like an operational program: clean local dev environments, full VMs where containers are not enough, strong permission boundaries, Slack and GitHub surfaces that behave like coworkers rather than tool endpoints, AI review that can push back, memory that retrieves without polluting context, and guardrails that prevent the codebase from decaying into accumulated agent slop.