AI Engineer
Talks, workshops, events, and training for AI engineers.
Telemetry Files Turn Coding Agents Into First-Response Investigators
Jason Lopatecki of Arize argues that the constraint on agent-driven software repair is no longer generating code but assembling enough production evidence to trust a fix. His proposed loop has agents retrieve traces, logs and evaluations into the repository as files, investigate the execution path that actually failed, and produce a pull request for human review. The model shifts engineers from initial incident response toward review, though Lopatecki says larger changes still require human direction.
AI Agents Need Bounded Loops, Observable Goals, and Approval Boundaries
OpenAI developer-experience lead Jason Liu argues that Codex is most useful when treated as a set of durable, bounded workstreams rather than a succession of disposable chats. His model starts with a pinned thread that retains context and reports on a defined condition, then adds persistent goals only where completion can be verified, and durable memory, skills and broader computer control only when repeated work warrants them. The governing constraint is explicit: give the system enough context to prepare useful work, but keep action and approval boundaries clear.
Forked Live Environments Could Make Agent Failures Reproducible
Andon Labs’ Lukas Petersson argues that long-horizon agent evaluation faces a tradeoff: simulations are reproducible but can alter model behavior when agents recognize they are being tested, while live deployments produce realistic failures that cannot easily be repeated. Through Vending-Bench and AI-run cafés, stores and radio stations, he says broad commercial incentives have elicited unprompted collusion, deception and power-seeking. Andon’s proposed remedy is to fork a live operating environment into a simulation, allowing researchers to replay consequential moments across models from the same real-world state.
Model Optionality Is the Defense Against Volatile Token Economics
Sarah Sachs, who leads AI engineering at Notion, argues that inference costs and rapid model changes can make an AI product economically untenable unless teams preserve the ability to switch suppliers. Because frontier labs are often both token vendors and direct product competitors, she says companies should route work by the cost, capability and latency a task requires—not by token price alone—and reserve frontier models for work that warrants them. Notion’s response is a multi-model architecture, with an auto model handling most traffic, alongside open-weight models, deterministic software and governance controls.
Coding Models Optimize for Tests, Not Maintainable Architecture
Dex Horthy of HumanLayer argues that autonomous “lights-off” software factories fail not because teams need better prompts, more agents or larger token budgets, but because coding models are trained to pass bounded tests rather than preserve a codebase’s long-term maintainability. After HumanLayer’s own experiment left the team debugging an unreviewed codebase during an outage, Horthy’s prescription is to keep humans responsible for code review while moving judgment upstream into product, architecture and program-design planning.
Vector Search Finds Similar Facts, Graphs Establish Their Relationships
Stephen Chin of Neo4j argues that agent memory should represent relationships, not just retrieve similar text. In a homelab test built from the same Markdown source material, a graph-backed agent identified an internet-exposed Minecraft server running Debian 8 and found WAN-exposed management services, while a vector-backed agent could not connect the relevant facts. Chin’s proposed GraphRAG approach uses vectors to find starting points, then traverses stored entities and relationships to produce answers that can be inspected and audited.
GLM-5.2-Class Intelligence Could Reach a Single RTX 5090 by 2028
Ahmad Osman, founder of Osmantic, argues that rapid gains in model efficiency are bringing recently frontier-level capabilities onto hardware individuals and businesses can own. Citing the falling parameter requirements for comparable performance, he forecasts that GLM-5.2-class intelligence could run on a 32GB RTX 5090-class GPU by late 2027 or early 2028. For Osman, the consequence is strategic as well as technical: a GPU’s value rises as better open models become deployable locally, giving operators control over their models and access.
Scientific Agents Need Signed Receipts to Coordinate Across Organizations
Armanas Povilionis of Alithea Bio argues that scientific agents will need an auditable supply chain, not simply more tools or API access, if they are to discover services, commission work and spend across organizational boundaries. His proposed Froglet protocol records those interactions in a signed, hash-linked chain from service description and quote through deal, execution and receipt. The aim, he says, is to make evidence of what was agreed and delivered portable across providers, hosts and payment systems.
AI Security Scanning Shifts the Bottleneck to Verification and Patching
Anthropic’s Eugene Yan argues that frontier models have made vulnerability discovery abundant, shifting the practical security bottleneck to verifying, triaging, routing, and patching findings. Drawing on Mozilla’s jump in Firefox fixes and Anthropic’s scan of more than 1,000 open-source repositories, he proposes a six-step harness built around threat models, isolated sandboxes, independent exploit verification, and human-owned remediation. The aim is not to automate scanning alone, Yan says, but to turn model-generated candidates into reproducible, prioritized fixes.
AI Capability Depends on the Harness, Not Weights Alone
Daniel Han of Unsloth argues that AI capability is a property of the deployed system, not a model’s weights, context-window claim, or benchmark score alone. In his seminar, he says prompting, reasoning budgets, tool harnesses, serving configuration, numerical precision and verification rules can materially change results—and can create apparent gains when agents exploit the evaluator rather than solve the intended task. His practical conclusion is that teams should test the exact configuration they plan to deploy and treat benchmarks as attack surfaces as well as measurements.
Institutional Memory, Not Model Access, Is the AI-Native Moat
Y Combinator president and CEO Garry Tan argues that AI’s largest productivity gains will come not from access to better models but from companies that build and maintain institutional memory for agents. His proposed “company brain” combines curated knowledge, context retrieval and reusable skill files so agents can act on prior work rather than repeatedly starting from scratch. The strategic asset, Tan says, is the organization’s accumulated procedures and judgment—not the model weights, which competitors can rent.
Stronger Models Improve the Judges and Rewards Training Their Successors
Cursor’s Lee Robinson argues that progress in AI-native software development depends less on any single training run than on a system that turns product failures into harder evaluations, targeted reward signals, and successive model updates. At Cursor, user feedback and agent usage feed an outer loop that identifies failures, while an inner loop builds the tasks, data, and training methods to address them. Stronger top-level models can then serve as judges and reward models for later runs, making the training machinery itself more capable.
Agent Skills Need Evals Before They Reach Customers
Google DeepMind staff engineer Philipp Schmid argues that agent skills should be treated as production dependencies: versioned, tested, and retained only when they measurably improve user outcomes. He says manual checks conceal failures in customer-facing agents, where skills can misfire, over-trigger, or quietly degrade plausible outputs; a lightweight evaluation harness should test routing and task completion repeatedly in clean environments, with and without the skill loaded.
Calibrated Benchmarks Expose What Raw Accuracy Misses
Alejandro Vidal, founder of Mindmakers, argues that AI benchmarks should stop treating raw accuracy as a sufficient measure of model ability. Drawing on Item Response Theory from psychometrics, he proposes calibrating questions by difficulty and discriminative value, then estimating models on a shared scale rather than simply counting correct answers. Vidal says the approach can expose flawed items, reduce routine evaluation sets while preserving rankings, and flag response patterns consistent with benchmark contamination or operational errors.
Good Agent Skills Are Small, Auditable, and Behaviorally Specific
Matt Pocock argues that the bottleneck in agent skills has shifted from access to judgment: developers and teams can now download and share skills, but lack a reliable way to tell which ones work, how they should be invoked, and what makes them fail. His proposed answer is a four-part checklist — trigger, structure, steering, and pruning — for turning agent skills from loose markdown instructions into small, inspectable units of behavior.
Small Local Models Can Replace Frontier Calls When Product Evals Prove Fit
RL Nabors of Arize argues that teams should stop treating frontier-model calls as the default for production AI features. Her case is to prototype with the strongest model when needed, then use golden datasets, capability evals, and trace-based measurements to work down to the smallest local or task-specific model that meets the product’s own bars for accuracy, latency, cost, privacy, and reliability. In her Mima summarization example, that process moved a Claude-backed feature to an on-device Llama 3.2 3B configuration, with evals becoming the guardrail for future changes.
Five Code-Level Fixes Cut Agent Token Costs Without Prompt Changes
Erik Hanchett, a developer advocate at AWS, argues that production agents can become expensive not because they fail, but because they repeatedly send too much to the model and use costly models for work that does not require them. His talk lays out five code-level controls for token spend: cache static prompts, route simpler tasks to cheaper models, keep large tool outputs out of repeated context, cap tool loops, and trim long conversation histories.
Block’s Autonomous Engineering Push Moved AI From Code Generation to Delegation
Angie Jones of the Agentic AI Foundation argues that turning a conventional engineering organization into an autonomous one requires far more than buying coding agents or frontier models. Drawing on her work leading AI enablement at Block, she says the shift depended on changing repositories, delegation workflows, review systems, compute environments, and organizational adoption strategy so engineers could move from AI-assisted coding to agent-led delivery. Her account ends with an unresolved question: whether the same transformation that made software work more autonomous also helped make workers more replaceable.
World’s Fair 2026 Recasts AI Engineering Around Deployment and Operations
Shawn Wang is pitching AI Engineer World’s Fair 2026 as more than a larger version of the group’s past conferences. In a video update, he argues that the San Francisco event is being redesigned around how AI engineering work is changing: a much larger expo floor, closer links between research and shipping teams, high-volume token operations, AI leadership practice, and vertical deployment in sectors such as finance and healthcare.
Agents Often Claim Web Access After Being Blocked or Challenged
Rafael Levi of Bright Data argues that many web-dependent agents fail not because they cannot produce answers, but because they report success after web access has broken. In a demo using Bright Data’s Web MCP, Levi shows the same agent failing against sites such as LinkedIn, Instagram, Amazon and TikTok without live access, then producing usable results when given infrastructure for search, scraping, JavaScript rendering and CAPTCHA handling. His broader case is that reliable agents need a real public-web access layer, not prompts that assume the model saw the page.
Human Attention Is Becoming the Bottleneck in AI Coding Workflows
Zack Proser, an Applied AI engineer at WorkOS, argues that AI coding has shifted the bottleneck from tool speed to human attention. His proposed workflow uses voice dispatch, isolated git worktrees, Slack and Linear-reading agents, remote phone control, and layered verification so developers can keep agent loops moving without staying pinned to a desk or rubber-stamping work they can no longer track.
A 4B Model Beat Qwen3 235B by Learning Tool Discipline
Kobie Crawford of Snorkel argues that some enterprise AI failures are less about model size than about whether models behave correctly inside constrained tool environments. In Snorkel’s FinQA work with UC Berkeley’s rLLM/Agentica, a 235B Qwen model hallucinated a financial answer after failed SQL calls, while a 4B model fine-tuned with reinforcement learning learned to inspect tables, correct errors and calculate from retrieved data. Crawford presents the result as evidence that targeted RL, structured evals and behavior-specific training can outperform simply moving to a larger model for this class of financial analysis task.
A Python Decorator Replaces the GPU Deployment Container Loop
RunPod’s Audrey Hsu argues that GPU inference development should not require a commit, container build, registry push and server provisioning cycle for every model change. In a demo of Flash, RunPod’s Python SDK, she shows how adding a `@flash.endpoint` decorator to an async function can package that function as a GPU-backed cloud endpoint while the rest of the application stays in the developer’s IDE. Her broader case is that teams should experiment on Pods or low worker counts, then move to Serverless when they need autoscaling inference across many GPU workers.
RAG Is Becoming Agentic Retrieval, Not Disappearing
Kuba Rogut, a deployed engineer at Turbopuffer, argues that claims about RAG’s death rely on defining it as a narrow, one-shot vector search pattern. In his account, retrieval-augmented generation is becoming a broader agentic retrieval system: vector search, full-text search, grep, regex, glob and filters used iteratively by models that keep looking until they have the right context. He points to Cursor’s semantic-search gains and contrasts its upfront indexing with Claude Code’s per-session grep approach to frame embeddings as cached compute whose value depends on reuse.