AI Engineers Are Building Systems That Evaluate and Improve Agent Work

? aparna-dhinakaran frames the changing role of an AI engineer through a funnel attributed in her presentation to Andrej Karpathy. The slide estimates that roughly 6 billion people have barely touched an LLM, 1–2 billion use one casually, and about 20 million are professionals in math and code, in a tier where an agent can finish work that might otherwise take weeks from a prompt. At the narrow end are about 5,000 people with frontier access, running swarms of thousands of agents.

Dhinakaran reads the funnel as a direction of travel: more engineers will need to learn how to operate many agents at once. She places that shift in a longer rise in abstraction. Engineers moved toward higher-level programming languages; more recently, they have moved from copying code out of chat assistants to using local agents and then cloud agents. The emerging expectation, she argues, is to run internal “factories of agents.”

That changes the unit of work. Instead of building a single AI feature, an engineer builds a system that can build, evaluate, ship, monitor, and improve features. Instead of personally reviewing every code diff, the engineer sets up agents to review code and writes evaluations for tasks whose results are not deterministic. And instead of relying only on software metrics or user complaints to spot problems, the engineer designs telemetry so agents can inspect it and surface patterns.

Dhinakaran describes the engineer’s time as shifting from writing and reviewing code toward defining what good looks like and reviewing the verdicts produced by automated systems. The responsibility is not only to perform each part of the development cycle, but to build the loop that runs those parts and helps determine whether the result can be trusted.

Arize built Signal to run part of this loop. Dhinakaran describes it as an always-on agent that reads application traces, identifies patterns in the telemetry, and can propose a fix or open a pull request if it has access to the repository.

A Signal run reads a configurable volume of traces on a cadence. It looks for possible issues, but it also needs a record of what it has seen before. An issue may have been flagged previously; a pattern that once seemed minor may look more serious as additional evidence arrives. Signal therefore preserves enough history to compare new observations with old ones, rather than treating each run as an isolated review.

Examples shown in the presentation include an agent claiming an action is complete without calling a tool, reporting a balance it never looked up, or pre-approving a request without checking a profile. These are failures inferred from the content and sequence of agent behavior in traces, not simply metric spikes.

Dhinakaran argues that observability data should increasingly be prepared for agents to read directly. In the earlier model, a human developer reads dashboards and traces. In a second stage, the human reads them and asks an agent to help write code. In the next stage, agents consume the observability data themselves. The data becomes an input to the agents investigating problems, as well as a record for human review.

For Signal, a run can start on a schedule—for example, every three or six hours—or in response to an event. Dhinakaran also lists Slack messages, user complaints, and metric-based triggers as possible ways to wake an agent. A trigger starts the run; Signal then gathers the context needed to investigate.

Each Signal run starts with a newly assembled working environment. Dhinakaran says the system gathers trace context and, where relevant, other telemetry such as application performance logs and metrics. It can also provide repository access and the keys needed to perform the task. That context is placed in a sandbox, with a proxy acting as the single route for information to enter or leave. Activity inside the sandbox is traced, and results—including issues found—are saved to memory.

The separation between what persists and what is discarded is deliberate. The output of a run, its traces, memory, and any useful new skills can be kept. The sandbox is torn down; credentials are removed; repository access and the temporary workspace are discarded. The presentation describes the memory as files rather than chat history. A later run can use the stored record of previous issues without inheriting the full temporary environment of the last one.

Dhinakaran also stresses that a run should survive failures. If a sandbox or the agent harness fails midway through the task, Signal should not have to restart from the beginning. It saves state outside the sandbox as each step finishes, including the conversation, memory, and the next step to take. A replacement sandbox and harness can then resume from that saved state.

Signal is also designed to work across models and harnesses. Dhinakaran says it began with Claude Code and supports Cursor and Codex, with the aim of supporting major open-source harnesses as well. Within a harness, the system can use different models. For Arize, this is both a portability choice and a way to compare quality and cost: a more capable model might produce better pull requests, but the question is whether a less costly option can retain the quality of issue detection.

If agents are going to review non-deterministic work, engineers need a way to test their judgments. Dhinakaran describes an evaluation that can begin simply: a dataset, a prompt, and outputs checked for regressions. One slide uses customer-support questions and answers. A judge is asked to label an answer correct or incorrect against a policy; in the example, an answer about a refund is marked incorrect because the policy says five to seven days, not 90.

But writing the evaluator is not enough. Dhinakaran says evaluations need to align with human judgment, and that alignment is a central part of the AI engineer’s work. Her suggested process starts with real traces rather than metrics alone: find the errors, name the ways the agent fails, turn those failures into a dataset and evaluation prompt, and check the evaluator against human labels.

That last step addresses a practical concern she hears from teams: how can they trust an evaluator to catch the same problems a person would? Arize’s approach is to evaluate the evaluators. The team builds a “golden truth” dataset, compares the evaluator’s outputs with human judgments, and works to bring them into agreement before shipping a new evaluator. Dhinakaran says these evaluations can be used both offline, during testing, and online, after a system is in production.

The engineer’s responsibility, then, is not merely to add an automated score to a workflow. It is to identify meaningful failure modes, create cases that represent them, and assess whether the evaluator’s judgments track human ones. The system’s ability to automate review depends on that work.

Dhinakaran extends the same approach to monitoring in production. Before shipping, teams can run offline evaluations against a proposed change. After shipping, they can evaluate live traces online. Those evaluations score agent behavior and help group failures into patterns that can become new evaluation cases or fixes.

That also means tracing the agents that build software, not only the agents that users interact with. Dhinakaran calls for tracing both coding agents—such as Claude Code, Cursor, and Codex—and user-facing agents. The aim is to understand what happened across the whole chain, including the coding agents behind the systems being deployed.

She sees this work as a way to narrow the gap between what frontier labs can demonstrate and what businesses can ship. Business data, she notes, may be messier than the constrained, testable settings in a lab. In her account, AI engineers help close that gap by tracing the systems they build, evaluating their behavior, and creating a process in which agents can use collected telemetry to improve the product.