Evals and Benchmarks
Methods for measuring model and system performance, including benchmarks, task-specific evals, red teaming, reliability tests, and quality measurement.
Verification Is the Bottleneck for Self-Improving AI Agents
Stanford’s Aakanksha Chowdhery and Azalia Mirhoseini argue that progress in AI is shifting beyond scaling model training toward using more computation at inference time, selecting among candidate outputs, and feeding verified successes back into training. In the opening lecture of CS329A, they trace how post-training made large language models usable assistants and how reasoning models and agents extend that capability into multi-step work. Their central constraint is verification: self-improvement is most tractable where systems can reliably tell whether an answer or action is correct.
LATS, SPRINT, and SWiRL Target Different Bottlenecks in Agent Reasoning
Stanford’s Azalia Mirhoseini argues that improving multi-step AI agents requires more than extending a chain of thought: agents must plan, act, use feedback and revise their course. In this CS329A lecture, she examines three approaches to that problem—LATS, which searches alternative action paths at inference; SPRINT, which identifies work that can run in parallel; and SWiRL, which trains next-action choices from offline tool-use trajectories. Each shifts the trade-off among accuracy, latency, inference cost and the reliability of feedback.
Longer Agent Task Horizons Still Fall Short of Dependable Autonomy
Stanford’s Aakanksha Chowdhery argues that longer agent task horizons should not be mistaken for dependable autonomy. Her CS329A lecture distinguishes among benchmarks that measure how long agents can complete tasks at given reliability levels, whether their outputs match professional work, and whether they can synthesize research with verifiable evidence. Across those measures, the central limitation is persistent: agents improve on well-specified tasks but remain unreliable when success requires recovery, organizational context, or judgment about what work needs doing.
Verification, Not Sampling, Limits Test-Time Scaling for AI Agents
Stanford’s Azalia Mirhoseini argues that repeated sampling improves reasoning-model performance only when a system can reliably identify correct candidates—and that verification cannot recover answers the generator never produces. In this CS329A lecture, she examines outcome-based and process-based reward models, rollout-generated labels and verifier ensembles as approaches to the gap between producing a correct solution and selecting it. The central systems question is how to spend a fixed inference budget among generating more candidates, using stronger or more numerous verifiers, and verifying each candidate more deeply.
Train-Time Scaling Makes Verifiable Reasoning More Consistent, Not Necessarily More Capable
Stanford adjunct professor Aakanksha Chowdhery argues that train-time scaling can make reasoning models more reliable by feeding verified outputs back into training, but only when a base model can already produce some useful solutions and sampled answers provide meaningful reward variation. In CS329A, she uses STaR, DeepSeekMath’s group-relative policy optimization and DAPO to show how self-generated rationales, relative rewards and RL stability techniques improve performance on verifiable tasks such as mathematics and code. The unresolved question, she says, is whether these methods expand a model’s problem-solving ability or chiefly make solutions it could already reach more likely to appear.
Test-Time Scaling Depends on Verifying Rare Correct Answers
Stanford computer scientist Azalia Mirhoseini argues that test-time compute can turn an existing language model into a more capable system by generating, revising and evaluating many candidate answers after a query arrives, rather than relying on a single response or additional training. The approach works best where outputs can be reliably verified—through formal checkers, unit tests or comparable signals—and where the value of better results justifies added cost and latency. But she cautions that extra inference cannot substitute for stronger pre-training on the hardest problems, and that weak verification can leave correct answers buried among more plausible wrong ones.
ElevenAgents Turns Customer-Service Procedures Into Monitored AI Workflows
ElevenLabs presents ElevenAgents as a platform for turning customer-service procedures into conversational agents that can complete defined tasks, from processing refunds to booking appointments, rather than merely answer questions. The company says teams can define workflows and system access in natural language, simulate them before release, then monitor production conversations through its Spotlight tool for operational problems and test proposed fixes. Configurable guardrails, private-cloud deployment and data-residency options are intended to constrain sensitive actions and support enterprise use.
Post-Training Lifts DeepSeek Flash Above Its Larger Pro Model
Two Minute Papers’ Karoly Zsolnai-Fehér argues that DeepSeek’s V4-Flash-0731 shows how much capability can be added through post-training rather than a larger base model. DeepSeek reports that the 384B-parameter model retains the prior Flash architecture and size but substantially improves on agentic and software benchmarks, including a 54.4 DeepSWE score versus 7.3 for Flash-Preview. Zsolnai-Fehér’s case is that improved planning, verification and error recovery—not new underlying capacity—account for the shift.
Chinese AI Safety Work Challenges the Case Against U.S. Safeguards
Nathan Labenz argues that China’s weaker safeguards and disclosure practices do not validate the American claim that frontier AI safety requirements are futile because Beijing will not slow its own developers. Reporting from China, he finds a university-centered safety ecosystem increasingly engaged with Western work on deception, evaluation awareness, interpretability and hazardous capabilities, alongside a state able to delay or constrain domestic deployments. China remains behind the leading US labs, Labenz says, but its record complicates a simplistic “but China” case against American safety obligations.
Hybrid Imitation Learning Builds Adaptive Parkour From 30 Seconds of Video
Researchers behind Hybrid Imitation Learning argue that a parkour controller can learn human-like movement from just 19 YouTube clips—about 30 seconds of footage—without becoming locked into replaying them. Their method trains a simulated character both to imitate motion in its obstacle context and to navigate new courses, producing stronger motion tracking than the tested baselines while retaining a reported 0.74 task-completion rate in noisy unseen scenes. The paper also shows the trade-off: a task-reward-only configuration completes more courses, and HIL can still fail or recover with unnatural poses.
Situational Awareness Cuts Leverage After 67% July Loss
Leopold Aschenbrenner says Situational Awareness remains an AI-focused hybrid fund after a July loss estimated at 67%, but has eliminated leverage, closed its shorts and sold part of its public portfolio to Citadel to avert a more damaging liquidity event. John Coogan and Tyler Cosgrove argue that the episode does not by itself disprove the AI trade, which Big Tech earnings continue to support, but exposes how concentrated, levered positions can become untenable amid sharp swings in AI-linked stocks.
Unlabeled Learning Depends on Assumptions About Hidden Structure
Stanford computer scientist Chris Ré argues that unsupervised learning can recover useful structure only by making explicit assumptions about what generated unlabeled data. In CS229’s treatment of k-means and Gaussian mixture models, he presents k-means as a local, least-squares method for compact clusters and GMMs as a probabilistic model of latent Gaussian sources, with EM alternating between estimated memberships and parameter updates to improve likelihood without guaranteeing a global optimum.
Zero Training Loss Does Not Eliminate the Need for Validation
Stanford computer science professor Chris Ré argues that modern models can interpolate their training data and still generalize, but that result does not weaken the case for disciplined evaluation. In this CS229 lecture, Ré frames generalization as the stability of a learning procedure across plausible training samples, explains how regularization can trade bias for lower variance, and insists that model selection belong on development data while a final test set remains protected from tuning.
The 1% Rule for Finding Durable AI Opportunities
Google chief scientist Jeff Dean argues that AI founders should test frontier models on the workflows they hope to build around: if a general model already succeeds meaningfully, rapid model improvement may erase the opportunity. More durable openings, he says, are tasks where the model succeeds only 0% or 1% of the time and where a small team can add private context, distinct data, specialized models or better evaluation loops. The larger question is whether that advantage will persist long enough—and whether solving the problem matters.
Automated AI Research Shows Early Gains in Models, Training, and Kernels
Richard Socher, CEO of Recursive AI, argues that science is constrained by the number of people able to work on increasingly specialized problems, and that agent systems could relieve that bottleneck by running the cycle of hypothesis, experiment, validation and revision at scale. His proposed “Eureka machine” would begin by automating AI research, using search, compute and software tools rebuilt for agents rather than human workflows. The benchmark gains he presents in model training, architecture search and CUDA kernels are early proof points, he says—not yet recursive self-improvement or autonomous science.
Science Lessons Should Start With a Phenomenon, Not a Formula
Former NASA engineer and YouTube creator Mark Rober argues that science instruction should begin with a phenomenon students want to explain, rather than with formulas or vocabulary they are asked to absorb. His free Class CrunchLabs curriculum for grades three through eight uses short videos and low-cost experiments to prompt investigation, with teachers leading the discussion and sense-making. Rober’s claim is not that screens can replace teaching, but that a stronger initial hook can help teachers overcome what he sees as a motivation gap.
Blue Finance Requires Revenue, Risk Sharing, and Verifiable Ecological Results
Ocean conservation lacks the dependable revenue streams that private investors typically require, Joan Larrea of Convergence argued, so blue finance must instead connect restoration to avoided losses, sovereign debt savings, supply-chain resilience or other measurable exposures. Larrea, insurance executive Liz Henderson and marine scientist Steven Kessel said debt swaps, catastrophe coverage and corporate-backed arrangements can direct capital toward coastal ecosystems, but only if they account for ecological interdependence, local implementation capacity and credible evidence of results.
AI Accelerates Exploits in the Patch Gap, Not Full Cyberattacks
Horizon3.ai chief executive Snehal Antani argues that frontier AI models are accelerating established offensive tasks—especially turning software patches into exploits while organizations remain unpatched—rather than enabling autonomous, end-to-end cyberattacks. But he says current agents are also unusually brittle in defended networks, clicking planted decoys far more often than human testers do. The defensive advantage, he argues, depends not on a “magic” AI tool but on fast patching, compensating controls, deception, containment and rehearsed incident response.
Kimi K3 Releases Open Weights at Frontier Coding Scale
Károly argues that Kimi K3 makes competitive frontier-level coding capability available under open weights, even if its 2.8-trillion-parameter scale puts local deployment beyond most users. The model’s reported benchmark results place it close to leading proprietary systems and first on the displayed SWE Marathon test, while Kimi’s technical report says it still trails the strongest proprietary models overall. His case is that releasing the weights and methods matters as much as the demos, enabling cheaper access, distillation and follow-on open-model work.
Stronger Models Require Smaller Agent Harnesses
Claude Code creator Boris Cherny argues that as models such as Opus 5 become more capable, AI products should remove inherited prompts, tools and workflow constraints rather than accumulate them. He says builders should test models on problems beyond their assumed limits, supply clear guardrails and ways to verify results, and use observed failures—not old model workarounds—to decide what to add back.
Telemetry Files Turn Coding Agents Into First-Response Investigators
Jason Lopatecki of Arize argues that the constraint on agent-driven software repair is no longer generating code but assembling enough production evidence to trust a fix. His proposed loop has agents retrieve traces, logs and evaluations into the repository as files, investigate the execution path that actually failed, and produce a pull request for human review. The model shifts engineers from initial incident response toward review, though Lopatecki says larger changes still require human direction.
Forked Live Environments Could Make Agent Failures Reproducible
Andon Labs’ Lukas Petersson argues that long-horizon agent evaluation faces a tradeoff: simulations are reproducible but can alter model behavior when agents recognize they are being tested, while live deployments produce realistic failures that cannot easily be repeated. Through Vending-Bench and AI-run cafés, stores and radio stations, he says broad commercial incentives have elicited unprompted collusion, deception and power-seeking. Andon’s proposed remedy is to fork a live operating environment into a simulation, allowing researchers to replay consequential moments across models from the same real-world state.
Model Optionality Is the Defense Against Volatile Token Economics
Sarah Sachs, who leads AI engineering at Notion, argues that inference costs and rapid model changes can make an AI product economically untenable unless teams preserve the ability to switch suppliers. Because frontier labs are often both token vendors and direct product competitors, she says companies should route work by the cost, capability and latency a task requires—not by token price alone—and reserve frontier models for work that warrants them. Notion’s response is a multi-model architecture, with an auto model handling most traffic, alongside open-weight models, deterministic software and governance controls.
Coding Models Optimize for Tests, Not Maintainable Architecture
Dex Horthy of HumanLayer argues that autonomous “lights-off” software factories fail not because teams need better prompts, more agents or larger token budgets, but because coding models are trained to pass bounded tests rather than preserve a codebase’s long-term maintainability. After HumanLayer’s own experiment left the team debugging an unreviewed codebase during an outage, Horthy’s prescription is to keep humans responsible for code review while moving judgment upstream into product, architecture and program-design planning.