A 91% Cache-Hit Rate Cuts One Agent Task From Over $200 to $21

George Cameron argues that choosing a model is only part of choosing what an agent will cost. For long-running tasks, the bill also depends on how many times the model is called, how much context each call takes in, and how much of that input qualifies for prompt caching. Those factors can make the same model dramatically more or less expensive to run.

Cameron uses Artificial Analysis’s AA-Briefcase benchmark to illustrate the point. It tests agents on knowledge-work tasks in an environment containing thousands of files, including work such as creating reports, presentations, and marketing videos. In the example he presents, Claude Opus 5.5 in Max reasoning mode costs an average of $21.05 per task. The cost build-up chart shows that the total is not explained by a single answer: the agent makes an average of 165 model calls to complete the task.

Each call has several cost components: answer tokens, reasoning tokens, and input tokens. The number of calls multiplies those costs across the whole trajectory. Input tokens have their own pricing distinction: Cameron says cache hits receive roughly a tenfold discount compared with uncached input.

In the Opus example, the chart reports 126 million input tokens across the task, with a 91% cache-hit rate. Cameron identifies input volume and cache-hit share as major drivers of a call’s cost; across the task, the number of turns is another decisive factor. If caching fails entirely, he says, the same task costs more than $200 rather than about $21.

Cameron says the figures change what builders need to track. For agent workloads, comparing models by output-token prices or reasoning-token counts alone is not enough: input tokens, cache-hit percentage, and turns per task also matter. He notes that caching may not be enabled in a particular inference setup, including a self-managed setup using vLLM. The model can remain the same while the task’s realized cost changes substantially with cache behavior.

Cameron’s AA-Briefcase comparison for Opus 5.5 shows that reasoning-effort settings can change task cost substantially. He says the Low setting can cost about a dollar per task, while Max costs $21.05. The comparison chart breaks out turns per task, average context per turn, input tokens per task, and cost. Its central point is that effort settings affect more than the number of reasoning tokens in a single call.

Cameron explains that reasoning effort can affect how many turns an agent takes. Those turns, in turn, carry message histories forward and can increase the input tokens accumulated over the task. In the chart, input-token use reaches 307,000 at Max, compared with 73,000 at High. The costs and token counts shown across settings make the point that the same model can generate very different end-to-end bills.

That is why Cameron cautions against treating reasoning effort as an isolated setting. Teams comparing effort levels need to look at task outcomes and total task costs, not assume the setting only changes the amount of reasoning in one response.

Cameron frames the cost gap as a reason to make deliberate model choices, not as a reason to default to either the most expensive or the cheapest option. In the cost-per-task chart, some leading models cost more than $5 for a task, while others cost $0.05 or less. He says the lower-cost models are less intelligent, but emphasizes that they can still be capable. As an example, he points to a model he calls “GPT-6 Luna,” correcting himself as he speaks, at seven cents per task; he says it is much better than GPT-4 was when it first appeared, and far better than GPT-3.5 Turbo.

A second chart compares Opus 5.5 with GLM-5.3-Flash on AA-Briefcase. Cameron says GLM-5.3-Flash, an open-weight model, costs an average of 21 times less per task. The chart’s cost bridge separates changes in turns, input and output token use, cache-hit share, and token prices. It shows turns falling from 165 to 83, input tokens from 3.6 million to 1.8 million per task, and output tokens from 0.6 million to 0.3 million. The share of input tokens that are cache hits falls from 92.5% to 80%, while the chart also shows lower input- and output-token prices for GLM-5.3-Flash. Cameron’s point is that all these drivers contribute to the cost difference, with input-token prices and cache-hit percentage especially consequential for long-horizon agent tasks.

For a general comparison of language models, Cameron describes the Artificial Analysis Intelligence Index as a synthesis of ten evaluations spanning knowledge work, scientific reasoning, coding and terminal use, and other considerations including knowledge and hallucination. He presents it as a practical guide to models for current and near-term applications, not as a measure of progress toward artificial general intelligence.

The index is a starting point, not a substitute for task-specific evidence. Artificial Analysis also publishes individual evaluation results. Cameron cites AA-AnalystAgent, a finance and accounting benchmark that asks models to complete challenging tasks in complex spreadsheets. He says its results show that models do not always follow their generalist ranking: Gemini models perform well on complex spreadsheets, as do Anthropic’s models.

Cameron also places the cost comparison in a fast-moving market. He says the number of model releases is increasing and describes progress as accelerating, which can make it difficult for developers to keep up. He says open-weight models have stayed within roughly three to nine months of leading proprietary models, making them an option in some cost decisions. For a particular task, Cameron’s benchmark examples point to examining specific capabilities as well as general scores.

Cameron’s cost comparison has a time dimension as well as a model dimension. Artificial Analysis groups models by intelligence score and tracks the cheapest available cost for reaching each level. In the example he describes, reaching an Intelligence Index score of 30 initially costs a little over a dollar per task. About a year later, the cost is around ten cents, and it has since fallen below that level.

Cameron estimates that access to a given intelligence tier can become roughly ten times cheaper within a year. He uses a hypothetical million-dollar agent workload to illustrate what that trend could mean: the same level of intelligence might cost $100,000, and potentially around $10,000 in two years.

His point for planning is that current costs may not remain a stable basis for long-term budgets. A production decision still has to meet current requirements, but Cameron advises keeping in view how quickly the cost of comparable intelligence may change.

Cameron recommends beginning model selection with an intelligence-versus-cost view and its Pareto frontier. In the chart, the frontier marks models that offer the strongest intelligence for a given cost, or the lowest cost for a given level of intelligence. The practical question depends on which constraint matters most: if the budget is fixed, find the strongest option within it; if the task requires a particular level of intelligence, look for the cheapest way to reach that level.

He divides tasks into two broad categories. For an “unlimited intelligence” task, more intelligence can continue to improve the result. His example is analyzing the cost drivers in a business area and deciding how to optimize them. The question, he says, is how much additional intelligence is worth in relation to budget, latency, complexity, and other trade-offs.

A “limited intelligence” task has a ceiling: once the model is capable enough, added intelligence does not improve the answer. Cameron’s example is asking how much was spent on subscriptions last month. If the agent has the right MCPs configured, he says, a cheaper model such as Llama and a more expensive model such as Astra could both return $80. In that case, the question is what level of intelligence the task needs and what the cheapest way is to get it.

The distinction is Cameron’s way of making the Pareto comparison concrete: a general intelligence ranking cannot by itself tell a team how much capability its task requires. Some work may justify paying for stronger reasoning; other work may have a clear ceiling. In either case, the relevant comparison is between the task’s requirements and the cost of meeting them.

Cameron recommends a four-step selection process: understand the landscape, compare models against priorities, shortlist candidates, and benchmark the finalists on the team’s own data and tasks. For prototypes or smaller workloads, he suggests testing the shortlist in the team’s own context and agent setup. For production decisions, he recommends building a custom benchmark, particularly given the scale of the cost differences he has described.

Public intelligence scores and evaluations help teams understand the options and narrow the field. To decide which model best fits a particular workflow, Cameron recommends testing finalists on the team’s own work and comparing task performance alongside time and cost per task.

Cameron presents Artificial Analysis’s Optima product as one way to build such a benchmark. He says users can provide context and work with the system to generate a benchmark, then compare models on task-specific performance and cost. The broader recommendation is to use general benchmarks to understand the available models, then test finalists against the work they will actually be asked to do.