NVIDIA’s Agent Infrastructure Vision Meets the Work of Deployment
Jensen Huang argues that AI infrastructure must support agents and the systems around them, not just model inference. Accounts from Arize AI’s Aparna Dhinakaran and ElevenLabs’ Mati Staniszewski show how deployment also depends on monitoring and evaluation, and on adapting interactions to people, languages and tasks.
The shift from AI models to agent systems
Jensen Huang’s GTC Berlin keynote makes the largest claim: AI infrastructure is moving from facilities built to run models toward “AI factories” designed to run agents. NVIDIA’s CEO presented a full-stack platform spanning hardware, libraries, open models and data-center systems as infrastructure for an economy in which companies and industries are built around AI. That is an industrial-scale vision; the keynote’s demonstrations and specifications show what NVIDIA is proposing, not how reliably agents will perform across real deployments.
In Huang’s account, an agent is more than a model served with a prompt. It also needs a sandbox, a harness, knowledge and tools; the harness cycles through context, reasoning, planning and action. The list shifts attention from the model alone to the conditions around it: where an agent runs, what it can access, and how it carries out work.
The Vera Rubin NVL72 DSX Pod’s displayed specifications included 25 exaFLOPS of AI. Huang used the rack-scale system and its full-stack platform to make the case for infrastructure at factory scale. He also presented OpenShell, with a gateway, supervisor and policy prover, as a platform for safer agent deployment. These are parts of NVIDIA’s proposed stack; Huang’s claims about partner adoption and the future market remain company claims and forecasts.
The keynote’s central distinction is consequential even apart from its projections: infrastructure designed to run agents has to support more than model inference. It has to accommodate the harness, tools and other components that let an agent act. That raises a practical question the infrastructure vision alone cannot answer: how do teams observe agent behavior, detect failures and judge whether a proposed correction is sound?
The operational loop around agents
Aparna Dhinakaran’s account at Arize AI takes up that operational question. As agents do more of the work, she argues, the AI engineer’s role shifts from building individual features toward building systems that can build, evaluate, monitor and improve them. Delegating execution does not delegate responsibility for deciding what good work looks like.
Arize’s Signal illustrates one approach. The always-on agent reads application traces, looks for patterns and can propose a fix or open a pull request when it has repository access. Dhinakaran showed examples where an agent claimed a task was complete without calling a tool, reported a balance it had never looked up, or approved a request without checking a profile. The traces let Signal examine the sequence of behavior behind a failure, rather than relying only on a metric or user complaint.
A run gathers relevant traces and, where useful, application logs and metrics in a sandbox. A proxy controls information entering and leaving that environment. Findings and useful memory can persist, while credentials and the temporary workspace are discarded when the run ends. Dhinakaran also described saving state as each step finishes so a replacement environment can resume if a run fails partway through. These boundaries, memory and recovery plans are not incidental details: they are part of how Arize proposes to make an automated investigation repeatable and contained.
But observing behavior is not the same as deciding whether an evaluator has judged it correctly. Dhinakaran’s process starts with real traces: identify failures, describe them, turn them into test cases, then compare an evaluator’s judgments with human labels. In her account, teams must assess the judge as well as the agent.
The figure comes from five cases shown in Dhinakaran’s presentation, not a general measure of evaluator performance. Its significance is the checking method it illustrates: automated review depends on engineers defining meaningful failure modes and testing whether the evaluator recognizes them. A score from an unverified judge can automate a verdict without establishing that the verdict is useful.
That work continues after deployment. Dhinakaran describes using offline evaluations before shipping and applying evaluations to live traces afterward. Failures observed in production can become new test cases or prompt a fix. She also argues for tracing coding agents as well as user-facing ones, so teams can inspect more of the chain that produces a deployed system.
Signal is Arize’s example of this approach, not proof that agent review is solved. Its design links monitoring, controlled access, retained context and recovery; the responsibility for defining acceptable behavior and checking proposed judgments remains with the engineers. That responsibility becomes more complex when “good” depends on how a system communicates with people, not just on whether it took a permitted action.
The human interface is part of the system
ElevenLabs CEO Mati Staniszewski makes the user-facing challenge concrete through voice AI in India. He argues that reasoning ability is only part of a product: people also need to understand an interaction and feel able to trust it. Staniszewski cited 22 official languages, 122 major languages and thousands of dialects and accents, with speech shaped by region and by the relationship between speakers. That variety makes language coverage one requirement among several, rather than a proxy for natural interaction.
A voice system must listen, understand context, respond at a suitable pace and convey the intended delivery. Staniszewski’s demonstrations included dubbing intended to carry a performance across languages and speech generation guided by directions such as “serious” or “whispering.” He said Eleven v4 supported 14 Indian languages, with plans to extend support to all 22. Those figures describe the company’s product and plans; they do not by themselves establish performance across every language or setting.
Timing is another part of the interaction. Staniszewski described Eleven v4 Turbo as optimized for conversational use, with a median latency of 100 milliseconds. For a voice agent, quick responses can help sustain a conversation, but speed is only useful alongside comprehension, context and appropriate behavior. These demands meet in the task: an exchange that works for a tutor may need different language, pacing and guardrails from one handling customer support.
The customer examples in Staniszewski’s keynote point to that variation. Pocket FM uses AI narration for long-form content across languages, while PhysicsWallah uses voice tutoring in Hinglish. Razorpay’s conversational agents support merchant engagement. Rather than demonstrating one universal configuration, the examples show voice applied to different jobs, each with its own expectations for delivery and context.
Staniszewski said ElevenAgents handles more than 15 million conversations a week, three times the volume of six months earlier. The company presents that as a measure of use. He also described forward deployed engineering teams working with customers to adapt systems to industry-specific terminology, guardrails and compliance requirements, calling production deployment “messy and unglamorous.” That account connects product performance to the work around the model: a speech system must be fitted to a task and the rules governing it.
This is where the interface challenge meets the evaluation challenge, without becoming the same problem. Teams need to determine whether an agent took an appropriate action and whether its response made sense to the person receiving it. The relevant standard can vary with language, industry and purpose. Natural-sounding speech may help, but it cannot stand in for those judgments.
Close on what deployment asks of the field
These accounts concern different companies and systems; they do not show a shared platform or coordinated strategy. Together, however, they make deployment a question of what must be built around an agent. Huang emphasizes infrastructure to run agents and the tools they use. Dhinakaran describes the monitoring, evaluation and recovery work needed to inspect and improve their behavior. Staniszewski shows how interaction has to fit the language, task and setting of the people expected to use it.
The demands vary by application, but none of the accounts treats the model alone as the whole product. The practical test is whether the surrounding systems—operational controls, reliable evaluation and a suitable human interface—can be made specific enough for the work at hand.