A Codex Agent Kept OpenAI’s CI Healthy for More Than 1,000 Hours

? aaron-friel gave Codex a recurring job: keep continuous integration healthy across OpenAI. CI runs automated tests so engineers and researchers can tell whether their changes are ready to merge. Some failures point to broken code; others are flaky tests that fail once and pass the next time. Working out which is which can take hours or days, while new failures keep arriving.

The agent, which Friel named Frielbot, ran for six weeks—more than 1,000 hours—before he stopped it. It was not because the system had stopped working: colleagues had started running their own agents on the same job.

The assignment did not specify how to fix each failure. Friel said the initial prompt was about 200 words, setting out the repository, asking the agent to work out how to run the test suite, and giving it rules for using CI resources and contacting colleagues. The agent had to investigate failures and work through them in a loop.

That approach contrasts with Friel’s method the previous year, when he used long, prescriptive prompts that specified libraries, supplied code samples and asked Codex to make a plan before starting. He said he is now more comfortable giving the model the goal, desired outcome and relevant tradeoffs, then letting it determine the steps.

The same shift appeared in Vaibhav Srivastav’s demonstrations. To build an interactive Ferrari model and website, Srivastav gave Codex seven reference pictures. Astra produced clay and wireframe views as well as the car model. For a Nürburgring fly-through, he gave Codex only the track’s name and asked it to find its own way to build the simulation.

Codex chose the longer Nordschleife circuit: 21 kilometres covering all 70 turns. Srivastav said Codex generated the fly-through using online references. The work took a little over two days, much of that time spent waiting for Blender to render frames. Codex used sub-agents to research the track, check geometry and plan the render. When RAM issues and CPU overheating interrupted the work, it resumed from saved frames.

Friel’s CI run put the outcome-led approach to work on a different kind of task: a stream of technical investigations that would otherwise keep demanding human attention.

At first, Frielbot investigated and fixed failures in one thread. Friel noticed that it could get absorbed in a particularly difficult problem while other flaky tests and failures accumulated. Four days into the run, he told it to delegate the work in parallel. The main thread then triggered tests, collected information about failures, organized sub-agents and coordinated with people. The sub-agents handled slower investigations: checking out code, working on an individual failure, pushing a change, waiting for CI, and repeating if necessary.

The shift turned the main agent into a coordinator for a team of agents. Over the run, Frielbot opened more than 100 pull requests per week, pushed more than 1,000 commits and sent more than 600 Slack messages to coordinate with colleagues. It merged more than 213 of its own fixes. When it found that someone else had already made a similar change, it sometimes prioritized that work instead.

Friel said a retrospective with the agent revealed that its most productive stretch lasted five days, during which he did not message it. As he put it, “I give an agent a job, and then it manages a team of agents to get it done for me.”

Friel reported just under half a million tool calls and more than 5,800 sub-agent sessions. Over 67,000 tool calls were spent waiting. In CI work, he said, people often push a change and leave to get coffee while a run completes. The agent could wait for the result and continue with the next step when it arrived.

The problems it addressed were not necessarily easy leftovers. Friel said many were unresolved issues that people had encountered and chosen to skip or retry rather than fix. One example was a nondeterministic error in floating-point calculations in a GPU test. The agent correlated logs across weeks and traced the intermittent failure to a particular machine. A sub-agent learned to run the test repeatedly on that node; reproducing the problem took several days. The agent then contacted the researchers responsible for the system to ask whether the cause might be an undeclared software optimization or a faulty GPU.

Getting CI green can also mean deleting a failing test. Friel said that when the agent was unsure whether this was the right solution, it sometimes used another agent to inspect and challenge the proposed fix before asking a human to review it. He said the work gave him back hundreds of hours over the summer, which he could spend scaling infrastructure.

Srivastav described six Codex features for supporting long-running work: a thread for each outcome, remote environments, plugins, child threads, Ultra and scheduled tasks. These sit alongside task-design practices such as supplying contextual instructions and keeping notes. Simply leaving an agent running is not enough. It needs a way to check its work, access to relevant context, and rules for when to proceed or ask for help.

For a CI task, Srivastav suggested giving one thread responsibility for monitoring failures, reproducing them, proposing fixes and opening pull requests for review. Tests provide a way to check whether a change worked. An AGENTS.md file can explain where tests live and what the agent should and should not do. His example instructions included running the failing test before and after a fix, recording what had been tried and what remained to investigate, and opening a pull request that states what was tested. They also said not to skip or disable a test merely to make CI pass, not to trigger unnecessary large CI runs, and not to merge the pull request.

Srivastav recommended having the agent keep notes on completed work and remaining questions, so it can resume if it cannot continue immediately. A long-running thread also has to retain useful context. He said Codex’s built-in compaction allows work to continue over long horizons without requiring users to change default behavior, and described having threads run for months.

The computing environment is another part of task design. A laptop is an inconvenient host for work that may run for days or weeks. Codex can connect to remote machines over SSH, making the machine’s files, repository and tools available across desktop, mobile and web. For the Nürburgring work, Srivastav used a development server with Blender, project files and web search. He also described Codex Cloud, which provides a virtual machine without requiring users to manage one. Its environment is snapshotted, and checked-out code dependencies refresh periodically; the same task can be accessed through desktop, mobile or web.

Access to background information matters too. If a test is failing, relevant context may be in a GitHub issue, a pull request, Linear or a team discussion. Srivastav argued that plugins can connect Codex to those systems so it can find related work rather than investigate without that context. He called plugins the strongest way to accelerate what users get out of Codex.

A single thread can become difficult to manage when a job has independent parts. Srivastav’s Ferrari example had both a Blender model and a website for exploring it. Since those pieces did not need to be completed sequentially, Codex could split them into child threads while a coordinating thread kept track of the overall goal. Each child thread could be opened, redirected or stopped independently. In a Git repository, separate worktrees give those threads their own checkouts, so they can make changes in parallel without interfering with one another.

For work that needs steering rather than splitting, Srivastav described /side as a temporary child thread that forks the main thread’s context. It lets a user ask questions about ongoing work without changing the main thread itself.

He distinguished parallel tasks from a single difficult problem. For “embarrassingly parallel” work, he would ask Codex to split the job across threads. For a hard problem such as the intermittent GPU failure, he would use Ultra, which gives the model more time to reason and can involve sub-agents investigating different approaches. In Srivastav’s framing, Ultra concentrates effort on one difficult problem; child threads divide work that can proceed independently.

Some tasks are blocked not by technical difficulty but by waiting for a person or another event. A scheduled task can check a pull request periodically, summarize review feedback and address what it can, leaving the rest for attention. Srivastav called a scheduled follow-up in the same thread a “heartbeat”: Codex returns with the context of the work so far and can continue from there. Scheduled tasks can also start recurring work, such as providing a morning update on open pull requests or a weekly summary of projects.

The six-week CI run was estimated at about $32 per hour, according to the speaker who answered the question about its cost. The speaker said the estimate was calculated by working backwards from pricing. GPT-6 Sol later brought the stated hourly figure down to $16; that was a pricing comparison, not a measured cost for the same six-week run. GPT-6.1 Sol was described as more token-efficient and competitive with Astra. The $32 estimate concerned an agent fixing bugs across OpenAI systems, repeatedly running CI and millions of tests as part of its loop. Code review, CI resources and other factors also contribute to the total cost.

The suggested starting point is recurring work that displaces more important work: reviewing a backlog, making speculative pull requests for roadmap features, watching logs for errors, or investigating CI failures. Give the agent a way to verify its work, such as end-to-end tests, and a clear signal for when the work is done. The speaker said that with the GPT-6 models, it was possible to specify the tradeoffs and desired outcome rather than prescribe every step.

The remaining controls belong in the assignment. Delegate independent work, match the model’s effort to the difficulty, and specify what matters and when a person should be involved. Set budgets too—whether that means money or, as in Friel’s CI task, limits on how many jobs can run at once. The agent can keep working, but the human still defines the acceptable tradeoffs, escalation points and review boundary.