Enterprise Agents Route Tasks Across Specialized And Frontier Models
NVIDIA argues that enterprise AI agents should route work across multiple models rather than rely on a single general-purpose system. In its example of Glean, a specialized model gathers context from enterprise sources and handles straightforward requests, while frontier models are used for complex analysis involving tools, multiple sources, or deeper reasoning. NVIDIA says this division can reduce latency and token use while preserving answer quality, though the performance figures are presented as Glean claims.
Route work to the model that fits the constraint
An enterprise agent should not send every request to the same model. Simple, high-volume, or sensitive-data tasks can go to smaller, specialized, or locally deployed models; frontier reasoning is reserved for work where deeper analysis changes the answer.
A request may require a quick policy answer, a search across internal records, a tool call, a multi-source synthesis, or strict control over where sensitive information is processed. NVIDIA positions these as different model jobs rather than variations of one universal task.
The architecture is a routed system: a user query enters a model router, which selects among models of different sizes and deployment types. The routing diagram distinguishes local from cloud models, while Glean’s wider system includes cloud frontier models, local models, custom models, and fine-tuned models alongside enterprise data sources and tools.
Frontier models are positioned for the hardest reasoning tasks. Open models serve a different role: they can be fine-tuned on company data for specialized work and run on-premises. NVIDIA says that local deployment can help keep sensitive data secure while reducing latency for high-volume tasks. The point is not that frontier and open models are interchangeable, but that they can be combined according to the task.
The same tradeoffs appear in the model-selection interfaces shown from Perplexity, ChatGPT, Claude, and Glean. Claude describes higher effort as more thorough but slower and more demanding of usage limits. Glean distinguishes “Fast” answers for single tasks from a “Thinking” mode for more complex work. Routing turns that choice into a system decision instead of requiring users to select a model manually for every prompt.
Glean gathers context before escalating to deeper reasoning
Glean is NVIDIA’s production example of a system-of-models approach for enterprise search and action. Its architecture begins with Waldo, a specialized model post-trained on NVIDIA Nemotron-3 Nano. Waldo gathers context across support tickets, Slack, and survey data; the displayed system also connects to document and web search. It then determines whether the agent can answer directly or should hand the task to a frontier reasoning model.
That ordering matters. When deeper reasoning is needed, NVIDIA says, the frontier model receives relevant enterprise context up front, allowing it to produce structured analysis with themes, evidence, and segment breakdowns more efficiently.
For a straightforward question, the depicted path is short. A request about home-office equipment reimbursement searches company knowledge and returns a policy-based answer in one response. The displayed answer says flexible-work home-office costs are generally not reimbursed, while eligible U.S. remote employees may receive a one-time $500 furniture stipend and may qualify for internet reimbursement of up to $50 a month. It also points to a separate process for medical accommodations and labels the output as AI-generated and potentially inaccurate.
In this route, Waldo supplies the context to a model that returns an answer in one shot. NVIDIA’s characterization is direct: the response is fast.
The complex route looks materially different. A request to compare approved public product materials with industry trends and customer-facing information passes through a frontier high-parameter model, a knowledge base, calculation tools, and data analysis before returning a specialized response. The displayed output explains its source constraints, compares capability themes against public trends, and identifies gaps in the available public-safe evidence.
The decision is therefore broader than document retrieval. The system must identify relevant context, then determine whether the request can be answered directly or requires deeper reasoning, analysis, and tools.
Routing is meant to cut latency and token use without sacrificing the answer
NVIDIA attributes Glean’s performance gains to routing enterprise context through an appropriate model path.
The displayed Waldo-on/Waldo-off comparison presents the associated claims together: 10 times faster search, 50% lower latency, and 25% fewer tokens, with no reduction in answer quality.
| Claimed result | Glean/NVIDIA claim |
|---|---|
| Enterprise-context search speed | 10× faster |
| Latency | 50% lower |
| Token use | 25% fewer |
| Answer quality | No reduction |
The side-by-side demonstration uses a request for updates on a Frasier Automotive issue. With Waldo on, Glean returns a synthesized update while the Waldo-off path remains loading. The response identifies a newly reported customer-facing 404 error, an unresolved high-priority data-analysis problem, missing historical content across service centers, and renewal risk. It recommends prioritizing the data-analysis issue and restoration of historical-content coverage over the newer 404 incident.
Glean’s depicted system is also broader than a choice between one small local model and one large cloud model. Its routing diagram names more than 30 models; cloud and local options; custom and fine-tuned models; enterprise sources including code bases and knowledge bases; and tools for email, databases, Slack, calculation, coding, planning, and sub-agents.
The operating principle is to use frontier reasoning where depth changes the result, and smaller, customized, or locally deployed models where retrieval, speed, privacy, or volume is the binding constraint. In NVIDIA’s account, an agent’s capability depends on matching the task, assembled context, model, and execution path—not on sending every request to the same model.