Phil Hetzel
Head of Solutions Engineering at Braintrust, where he works on AI evaluation and observability for production AI systems. He has spoken at AI Engineer Europe on generative AI team ownership and the challenges of building eval platforms for AI agents.
Gigabyte-Scale Agent Traces Are Forcing a New Observability Stack
Phil Hetzel of Braintrust argues that agent observability is a different problem from traditional observability because the central question is no longer whether a system is up, but whether an agent did the right thing. In his account, agent traces are too large, textual, and semantically loaded for uptime-oriented monitoring systems: Braintrust has seen traces exceed a gigabyte and spans reach 20 megabytes. Hetzel says that shift also changes who uses the data, bringing clinicians, lawyers, wealth advisers, and other domain experts into trace review so their judgments can become inputs for automated scoring and evaluation.
Agent Evals Should Replay Production, Not Exhaustively Imitate Unit Tests
Phil Hetzel of Braintrust argues that teams should stop treating evals for AI agents like unit tests meant to cover every possible failure. His maturity model starts with human judgments that record why an output failed, turns those justifications into scalable scorers, and then uses production traces to drive offline experimentation. The hard edge, he says, comes with tool-using agents, where useful evals must account not just for the final answer but for external system state and side effects at the moment the trace originally ran.
Enterprises Are Misassigning GenAI Work to Traditional ML Teams
Phil Hetzel of Braintrust argues that many enterprises misassigned generative AI work to data science and ML platform teams because it carried the AI label. His case is not that those teams are irrelevant, but that LLM application work starts after providers such as OpenAI and Anthropic have trained the base models. What remains, he says, is a broader product and systems problem: prompt and context engineering, domain annotation, functional evaluation, observability, and production feedback loops that require data scientists, engineers, and subject-matter experts working together.