AI Research Methods
New technical methods, papers, architectures, training techniques, reasoning approaches, and research findings with applied significance.
Supervised Learning Depends on Data Representing the Deployment World
Stanford computer science professor Chris Ré presents supervised learning as a wager that labeled training data represents the world in which a model will be deployed. Using linear regression, he shows how model design, feature choice and loss functions turn that wager into an optimization problem; using stochastic gradient descent, he argues that large-scale training accepts noisy minibatch updates in place of exact full-dataset calculations. The resulting batch choices are not merely computational: their size, order and composition can affect the predictor a system ultimately learns.
Exponential Families Link Likelihood, Inference, and Multiclass Classification
Chris Ré’s CS229 lecture argues that exponential-family distributions provide a common statistical framework for prediction, inference and maximum-likelihood learning, while generalized linear models add the assumption that a target distribution’s natural parameter is a linear function of features. In that formulation, logistic regression, least squares and multiclass softmax differ mainly in their output distributions; cross-entropy follows from the likelihood, and label smoothing relaxes the assumption that training labels are perfectly certain.
Battery Storage’s Next Bottleneck Is Market Design
Battery storage has become practical grid infrastructure, but its further expansion will depend less on cell costs than on whether power markets and regulators pay for the capacity, congestion relief and reliability it provides, said Jupiter Power’s Jack Godshall. Falling costs have made storage deployable, but scaling it requires systems that can value its services, supply equipment affordably and operate a growing fleet. Chibueze Amanchukwu of the University of Chicago argued that sodium-ion and other earth-abundant chemistries illustrate the trade-off: they could lower costs, improve safety and reduce supply-chain dependence, but must compete with increasingly cheap lithium-ion.
Kimi K3 Releases Open Weights at Frontier Coding Scale
Károly argues that Kimi K3 makes competitive frontier-level coding capability available under open weights, even if its 2.8-trillion-parameter scale puts local deployment beyond most users. The model’s reported benchmark results place it close to leading proprietary systems and first on the displayed SWE Marathon test, while Kimi’s technical report says it still trails the strongest proprietary models overall. His case is that releasing the weights and methods matters as much as the demos, enabling cheaper access, distillation and follow-on open-model work.
AI’s Broad Access Will Depend on Concentrated Compute Infrastructure
Sam Altman argues that OpenAI’s task is to make advanced AI as broadly available as electricity while building the concentrated compute, energy and data-center infrastructure required to produce it. He says demand for cheap, capable intelligence could be effectively uncapped, making large-scale inference revenue the basis for ever-larger training runs. But Altman also warns that cyber risks and the concentration of frontier capabilities could undermine the human agency that, in his account, widespread AI is meant to expand.
Coding Models Optimize for Tests, Not Maintainable Architecture
Dex Horthy of HumanLayer argues that autonomous “lights-off” software factories fail not because teams need better prompts, more agents or larger token budgets, but because coding models are trained to pass bounded tests rather than preserve a codebase’s long-term maintainability. After HumanLayer’s own experiment left the team debugging an unreviewed codebase during an outage, Horthy’s prescription is to keep humans responsible for code review while moving judgment upstream into product, architecture and program-design planning.
Caregiving Participation Is Linked to Lower Testosterone in New Fathers
Anthropologist Anna Machin argues that among biological fathers studied in early parenthood, testosterone tends to fall in connection with active caregiving—not simply with becoming a father. She frames the change as an adaptive shift from mating competition and risk-taking toward pair bonding, patience and infant care, while cautioning that the effects of testosterone supplementation on fathers remain poorly studied. Her broader case is that masculinity should be judged by responsiveness to changing roles, not by maintaining permanently high testosterone.
NVIDIA Cosmos Uses Compute to Generate Training Data for Physical AI
NVIDIA argues that physical-AI developers cannot collect enough real-world data to cover an “infinite and unpredictable” environment, and that compute must therefore generate the missing training material. Its Cosmos model combines an autoregressive reasoner with a diffusion generator to process and produce pixels, actions, sound and language, generating physics-aware scenarios and action-conditioned futures. NVIDIA says post-training adapts the model into a world action model for particular vehicles, robots and other physical systems.
World Models Aim to Replace Robotic Trial and Error
Ankit Gupta and François Chaubard argue that AI needs world models—systems that predict how an environment will change after an action—to escape the poor sample efficiency of trial-and-error learning. They contend that passive video could provide broad knowledge of physical change, then be paired with smaller action-labeled datasets to train robots in simulated rollouts. But Chaubard says the approach still faces hard limits in large action spaces, rare safety-critical events, real-time planning, and adaptation when physical conditions differ from the model’s predictions.
Cross-Entropy Measures the Cost of a Mismatched Probability Model
Grant Sanderson of 3Blue1Brown defines cross-entropy as the average coding cost of data generated by one probability distribution when encoded using another distribution’s assumed probabilities. In language-model pre-training, that cost is the average negative log probability assigned to observed next tokens, so minimizing it pushes the model toward the distribution underlying its data. In distillation, a smaller model minimizes cross-entropy against a larger model’s full probability distribution; KL divergence is the excess coding cost above the entropy of a matched code.
Stronger Models Improve the Judges and Rewards Training Their Successors
Cursor’s Lee Robinson argues that progress in AI-native software development depends less on any single training run than on a system that turns product failures into harder evaluations, targeted reward signals, and successive model updates. At Cursor, user feedback and agent usage feed an outer loop that identifies failures, while an inner loop builds the tasks, data, and training methods to address them. Stronger top-level models can then serve as judges and reward models for later runs, making the training machinery itself more capable.
Token Prediction Produces an Emergent Line-Length Counter
Anthropic researchers Gurnee, Ameisen and Batson argue that a language model trained only on token sequences learned an internal, approximate way to track line length and predict whether a word will fit before a line break. Their analysis traces that decision to features representing token length, position, inferred line width and distance from the boundary, arranged not as a simple counter but as a curved, spiral-like geometry analogous to biological place cells.
Long Context Cannot Replace Persistent Learned Memory
Engram co-founder and CEO Dan Biderman argues that larger context windows and retrieval systems cannot give AI the accumulated understanding needed to work reliably across an organization’s expanding knowledge base. Models that must reread documents, histories and decisions on every query are costly and prone to “context rot,” he says; Engram’s answer is a trainable memory layer that compresses recurring knowledge into learned representations. The longer-term challenge is deciding what a model should internalize, keep in explicit records and update over time without degrading performance.
Calibrated Benchmarks Expose What Raw Accuracy Misses
Alejandro Vidal, founder of Mindmakers, argues that AI benchmarks should stop treating raw accuracy as a sufficient measure of model ability. Drawing on Item Response Theory from psychometrics, he proposes calibrating questions by difficulty and discriminative value, then estimating models on a shared scale rather than simply counting correct answers. Vidal says the approach can expose flawed items, reduce routine evaluation sets while preserving rankings, and flag response patterns consistent with benchmark contamination or operational errors.
InfiniteDiffusion Decouples Terrain Generation Cost From World Size
Alexander Goslin’s InfiniteDiffusion argues that learned terrain generation need not slow down as a virtual world expands. Presented by Two Minute Papers’ Karoly Zsolnai-Fehér, the system generates each new area from overlapping local windows rather than modelling the entire map at once, while using a multiscale height-map method intended to retain both large landforms and small features such as creeks. Goslin reports training the model largely on a single RTX 3090 Ti and has released it as code and a Minecraft mod.
Codex 5.6 Helped Disprove a Three-Year Algebraic Surfaces Conjecture
Bartosz Naskręcki, a computational mathematician at Adam Mickiewicz University and CCAI Warsaw, says OpenAI’s Codex 5.6 helped his team break open a three-year problem in algebraic surfaces by finding a route they had not tried. The result was not the proof they had been seeking, but a disproof: in his account, the model’s value was in organizing and executing enough technical computation to make a difficult mathematical discovery reachable.
J-Space Gives Researchers a Readable Workspace Inside Language Models
Nathan Labenz and Prakash Narayanan assess Anthropic’s J-space work as a potentially important interpretability advance: a computed Jacobian lens that appears to expose a model’s internal workspace rather than merely its written chain of thought. Labenz argues the finding could matter for AI safety because the space seems causally tied to flexible planning, hidden objectives and strategic reasoning, while Narayanan treats it more cautiously as a useful but incomplete tool whose larger claims remain unsettled.
DeepSeek Reports 60%–85% Faster Per-User Generation With DSpark
DeepSeek’s DSpark paper argues that large language models can be made faster at inference without making them more capable, by reducing waste in speculative decoding. Károly Zsolnai-Fehér presents the technique as a serving-side change: a smaller draft model proposes tokens, the larger model verifies them, and DSpark improves the process through short-range draft memory, early rejection of weak continuations, and confidence-based scheduling. The practical claim is a reported 60% to 85% per-user generation-speed gain over DeepSeek’s MTP-1 baseline, not the 661% throughput figure shown in exceptional cases.
Claude Appears to Use a Small Readable Workspace for Reasoning
Anthropic argues in a research article that Claude appears to have a small, readable internal workspace: a set of word-linked representations separate from both its visible output and its broader automatic processing. Borrowing from global workspace theory in neuroscience, the company says this “J-space” can support hidden reasoning, be steered only imperfectly, and sometimes reveal concepts the model is not saying aloud. Anthropic says the finding does not show Claude is conscious, but does suggest that some important model behavior passes through an accessible intermediate layer.
GLM 5.2 Narrows the Open-Weight Gap With Frontier AI
Károly Zsolnai-Fehér argues that Z.ai’s GLM 5.2 matters less as a benchmark challenger than as evidence that open-weight AI is closing in on proprietary frontier systems. He says the model is not yet at the level of Claude Opus, Mythos or Fable, but its rapid gains in long-horizon coding and agentic tasks make ownership the central issue: whether users can download, run and keep powerful models rather than depend on systems that can be restricted, degraded or rerouted by their providers.
AI Math Progress Is Jagged, Not a Clean AGI Benchmark
Grant Sanderson argues that AI’s rapid gains in mathematics are less a clean proxy for AGI than a map of uneven capabilities: solving contest problems, finding cross-field connections, inventing definitions, verifying proofs, and explaining ideas are different tasks with different signals. In conversation with Dwarkesh Patel, Sanderson says the most consequential mathematical breakthroughs may be the hardest for current systems to learn, because their value often depends on delayed judgment, human taste, and concepts that compress a field rather than merely prove a theorem.
Scientific AI Is Moving From Single Models to Agent-Native Environments
At Stanford’s 2026 Conference on Physics and AI, James Zou argued that scientific AI is moving beyond stronger individual models toward systems of many agents and the environments that let them work together. He presented Stanford projects including the Virtual Lab, Virtual Biotech, EinsteinArena, Paper2Agent, and Paperclip as evidence that progress may depend on organizing agents, verifiers, incentives, and agent-readable knowledge structures as much as on improving model capability itself.
Physics AI Work Depends on Rigorous Problems and Interdisciplinary Collaboration
Risa Wechsler used her opening remarks at Stanford’s 2026 Conference on Physics and AI to argue that physics is well placed to make serious use of AI because many of its hardest problems are already rigorous, data-driven and computational. Speaking as director of KIPAC and Stanford’s Center for Decoding the Universe, she framed the opportunity less as a broad endorsement of AI than as a call for grounded collaboration between physicists, data scientists, computer scientists, engineers and statisticians.
AI’s Next Training Paradigm Depends on Learning From Deployment
Dwarkesh Patel argues that frontier AI labs are betting too much on reinforcement learning from verifiable rewards: training models across vast numbers of checkable, replayable tasks in the hope that this produces general agents. In his account, verifiability is not enough; the domains that matter most are often too slow, messy, and non-repeatable to be “grindable” training environments. The next paradigm, Patel suggests, will depend on whether models can turn scarce deployment experience into durable updates to their weights, through continual learning methods such as on-policy self-distillation, “dreaming,” or something not yet invented.