AI Application Architecture
Patterns for building production AI applications, including orchestration, memory, tool use, state management, APIs, and system design.
Long Tool-Use Trajectories Expose Self-Distillation’s Stability Limits
Ronak Malde of Trajectory argues that on-policy self-distillation can make production agent traces usable as dense training signal: a model learns from its own trajectories by matching a version of itself given privileged hints. He says the method avoids the parallel rollouts and trajectory-level rewards of GRPO, but breaks down on long, tool-using tasks when the teacher repeatedly corrects a student that has drifted off course, producing what he calls the “but wait” problem. Malde’s proposed remedies—step-level divergence weighting and residual guidance—are meant to preserve useful correction without teaching the model to hedge or exploit answer-revealing hints.
AI Product Differentiation Is Shifting From Interfaces to Intelligence
Sequoia Capital partner Sonya Huang argues that companies should decide which AI capabilities to own down to the model weights and which to rent from frontier providers, rather than treating sovereignty as an all-or-nothing choice. Her test is whether cost, latency, domain-specific performance and proprietary data make intelligence central enough to control. As open-weight models approach frontier performance, Huang says application companies can use their data, evaluations and production feedback to build specialized systems that outperform general APIs in their domains.
The Emergency Docket Is Reshaping How the Supreme Court Decides Cases
Neal Katyal and Katherine Wellington argue that persuading the Supreme Court requires more than doctrinal command: advocates must identify the particular textual, historical or institutional concern that may move individual justices under pressure. Their account of recent cases—from challenges to presidential tariffs and jury discrimination to gun regulation—also portrays a Court whose ideological divisions do not always determine outcomes, but whose turn from precedent and growing use of emergency orders have made both advocacy and judicial review less predictable.
Context, Permissions, and Network Overhead Define Agent Performance
OpenAI’s Dominik Kundel argues that an agent’s practical capabilities depend as much on its harness as on its model: the surrounding system determines how context is assembled, which tools are available, what actions require review, and how work persists across turns. In Codex, that means deferring tool definitions to protect context, matching file and shell interfaces to model training, and using constrained review agents for sandbox escalations. As inference accelerates, Kundel says, the harness must also reduce network overhead through stateful WebSocket transport and manage long-running work through explicit goals and context compaction.
Stable Prompt Prefixes Cut Long AI Agent Input Costs
Alejandro AO argues that long-running AI agents become expensive because every turn reprocesses an expanding transcript, not just the latest user input. Prompt caching can sharply reduce that input bill by discounting previously processed prompt prefixes, he says, but only if an agent harness keeps system instructions and conversation history stable and append-only. Dynamic material placed early in a prompt, compaction, cache expiry and unenabled provider settings can all reset those savings.
Character Reference Sheets Make AI Film Workflows Reusable
ElevenLabs’ Alec Wilcock argues that AI short films should be built around continuity rather than motion: establish the character, visual style and composition in still images before spending credits on video. His *Little Wanderer* workflow uses a reusable character reference sheet, narration-led story beats, and start and end frames to constrain each shot, then assembles and trims the results in Studio. The node-based Flows canvas, he says, makes the project reusable by allowing an upstream character change to regenerate the same film structure with a new protagonist.
OpenAI’s First Device Tests the Case for Personal AI Systems
The article argues that AI’s value may accrue less to models than to the products and compute systems that make their capabilities usable, while the costs of new technologies may be shifted onto the public. Coogan treats OpenAI’s reported device and Google’s infrastructure strategy as tests of that proposition; AI-designed bacteriophages raise a separate question of whether cheaper biological design can widen medical uses while creating biosecurity risks. The hosts also point to social-media litigation as a potential model for making platforms bear public costs attributed to product design.
Figma Positions Design as the Operating Layer for AI-Built Software
Figma chief executive Dylan Field says the company is investing in AI agents and code generation on the premise that cheaper, easier-to-generate code will make design the layer where teams define and steer software. Figma is already monetizing AI but is not including unproven products in its forecast, even as it reports accelerating revenue growth and a negative operating margin. Its aim is to connect design and code in the Figma canvas, allowing teams to build from design rather than treating it as an upstream handoff to engineering.
Agent Plugins Standardize Packaging for Skills and MCP Servers
Corey Ching presents Agent Plugins as an open specification for packaging agent skills and MCP server configurations in a format that can be discovered across products. Developed with AWS, Cursor, GitHub, Microsoft, OpenAI and Vercel, the standard defines a root manifest and common directory structure while leaving runtimes, permissions, marketplaces and client-specific interface choices to individual tools.
Enterprise Agents Route Tasks Across Specialized And Frontier Models
NVIDIA argues that enterprise AI agents should route work across multiple models rather than rely on a single general-purpose system. In its example of Glean, a specialized model gathers context from enterprise sources and handles straightforward requests, while frontier models are used for complex analysis involving tools, multiple sources, or deeper reasoning. NVIDIA says this division can reduce latency and token use while preserving answer quality, though the performance figures are presented as Glean claims.
Verification Is the Bottleneck for Self-Improving AI Agents
Stanford’s Aakanksha Chowdhery and Azalia Mirhoseini argue that progress in AI is shifting beyond scaling model training toward using more computation at inference time, selecting among candidate outputs, and feeding verified successes back into training. In the opening lecture of CS329A, they trace how post-training made large language models usable assistants and how reasoning models and agents extend that capability into multi-step work. Their central constraint is verification: self-improvement is most tractable where systems can reliably tell whether an answer or action is correct.
LATS, SPRINT, and SWiRL Target Different Bottlenecks in Agent Reasoning
Stanford’s Azalia Mirhoseini argues that improving multi-step AI agents requires more than extending a chain of thought: agents must plan, act, use feedback and revise their course. In this CS329A lecture, she examines three approaches to that problem—LATS, which searches alternative action paths at inference; SPRINT, which identifies work that can run in parallel; and SWiRL, which trains next-action choices from offline tool-use trajectories. Each shifts the trade-off among accuracy, latency, inference cost and the reliability of feedback.
Test-Time Scaling Depends on Verifying Rare Correct Answers
Stanford computer scientist Azalia Mirhoseini argues that test-time compute can turn an existing language model into a more capable system by generating, revising and evaluating many candidate answers after a query arrives, rather than relying on a single response or additional training. The approach works best where outputs can be reliably verified—through formal checkers, unit tests or comparable signals—and where the value of better results justifies added cost and latency. But she cautions that extra inference cannot substitute for stronger pre-training on the hardest problems, and that weak verification can leave correct answers buried among more plausible wrong ones.
ElevenAgents Turns Customer-Service Procedures Into Monitored AI Workflows
ElevenLabs presents ElevenAgents as a platform for turning customer-service procedures into conversational agents that can complete defined tasks, from processing refunds to booking appointments, rather than merely answer questions. The company says teams can define workflows and system access in natural language, simulate them before release, then monitor production conversations through its Spotlight tool for operational problems and test proposed fixes. Configurable guardrails, private-cloud deployment and data-residency options are intended to constrain sensitive actions and support enterprise use.
MCP Apps Standardizes Host-Controlled UI Across AI Assistants
Liad Yosef and Ido Salomon argue that MCP Apps gives AI assistants a way to render service-specific interactive interfaces rather than reducing every result to text. Built from the MCP-UI project and being developed as an MCP extension, the approach lets a server return an app resource that a compatible host renders in a sandbox, while clicks and other actions return through the host for further orchestration. They present it as a portability and distribution model, though state management and interoperability remain active work.
Tau Ports Pi’s Coding-Agent Architecture to a Textual Terminal
Alejandro Ao presents Tau as a Python implementation of Pi’s coding-agent harness, built to preserve Pi’s parent-linked sessions, core tools, skills, and event design rather than introduce a different agent architecture. He argues that Tau’s main departure is its Textual terminal interface, which makes a run’s context, tool activity, resource provenance, branching history, and exports visible alongside the conversation.
Automated AI Research Shows Early Gains in Models, Training, and Kernels
Richard Socher, CEO of Recursive AI, argues that science is constrained by the number of people able to work on increasingly specialized problems, and that agent systems could relieve that bottleneck by running the cycle of hypothesis, experiment, validation and revision at scale. His proposed “Eureka machine” would begin by automating AI research, using search, compute and software tools rebuilt for agents rather than human workflows. The benchmark gains he presents in model training, architecture search and CUDA kernels are early proof points, he says—not yet recursive self-improvement or autonomous science.
Stronger Models Require Smaller Agent Harnesses
Claude Code creator Boris Cherny argues that as models such as Opus 5 become more capable, AI products should remove inherited prompts, tools and workflow constraints rather than accumulate them. He says builders should test models on problems beyond their assumed limits, supply clear guardrails and ways to verify results, and use observed failures—not old model workarounds—to decide what to add back.
NVIDIA Built Its Strategy Around Algorithmic Domains, Not Chips
Jensen Huang argues that NVIDIA’s strategy has never been simply to build better chips, but to identify algorithmic domains where new computing architectures can change what is feasible and then build the stack around them. Recounting NVIDIA’s early failure in 3D graphics, its interpretation of deep learning after AlexNet, and its push into agents and robotics, Huang makes the case that technical leadership depends on confronting wrong assumptions quickly, learning the underlying workload, and organizing close to the work.
Lower Qubit Requirements Narrow Quantum Computing’s Path to Utility
Yaqumo CEO Kazuhiro Nakashoji argues that quantum computing’s long-delayed commercial timeline is narrowing because hardware capabilities are rising as the number of qubits needed for useful work falls. He says progress in algorithms, error correction and neutral-atom hardware could bring those curves together around 2030, though quantum processors would serve as specialized accelerators alongside conventional systems rather than replace them. Yaqumo’s bet is that integrating the lasers, vacuum equipment and software behind those machines can turn laboratory research into deployable infrastructure.
AI Agents Need Bounded Loops, Observable Goals, and Approval Boundaries
OpenAI developer-experience lead Jason Liu argues that Codex is most useful when treated as a set of durable, bounded workstreams rather than a succession of disposable chats. His model starts with a pinned thread that retains context and reports on a defined condition, then adds persistent goals only where completion can be verified, and durable memory, skills and broader computer control only when repeated work warrants them. The governing constraint is explicit: give the system enough context to prepare useful work, but keep action and approval boundaries clear.
AI Agents Assemble Documentary Motion Graphics From a Structured Brief
ElevenLabs presents a workflow for producing scrapbook-style documentary videos in which a tested visual prompt, persistent background and repeated reference assets establish continuity before scene generation begins. Claude converts a script and creative direction into a production brief, while ElevenCreative Flows builds the canvas, generates storyboard options and animates selected scenes. The company’s argument is that agent-assisted production can reduce manual setup without removing the creator’s role in supplying assets, choosing variants and revising the edit.
Model Optionality Is the Defense Against Volatile Token Economics
Sarah Sachs, who leads AI engineering at Notion, argues that inference costs and rapid model changes can make an AI product economically untenable unless teams preserve the ability to switch suppliers. Because frontier labs are often both token vendors and direct product competitors, she says companies should route work by the cost, capability and latency a task requires—not by token price alone—and reserve frontier models for work that warrants them. Notion’s response is a multi-model architecture, with an auto model handling most traffic, alongside open-weight models, deterministic software and governance controls.
Coding Models Optimize for Tests, Not Maintainable Architecture
Dex Horthy of HumanLayer argues that autonomous “lights-off” software factories fail not because teams need better prompts, more agents or larger token budgets, but because coding models are trained to pass bounded tests rather than preserve a codebase’s long-term maintainability. After HumanLayer’s own experiment left the team debugging an unreviewed codebase during an outage, Horthy’s prescription is to keep humans responsible for code review while moving judgment upstream into product, architecture and program-design planning.