AI Research Agents Are Advancing First Where Results Can Be Checked Quickly

Richard Socher sees AI’s role in science as moving beyond writing code, analyzing data, and drafting papers. The harder step is getting systems to propose ideas, carry them through experiments, and tell whether the results hold up. Socher says this is already happening at small scales, but not as a sudden transition to automatic science. Recursive self-improvement still needs suitable environments and substantial computing power; more complex problems require more compute. He rejects the idea that self-improvement by itself is a “magic bullet” after which everything else falls into place.

His starting point is that AI has advanced fastest where systems can draw on abundant data, simulate outcomes, or verify answers. Language has a vast store of human-produced text. Games can be played repeatedly in simulation, with a clear win-or-loss signal. Mathematical proofs can, in principle, be checked. Those conditions let systems learn from many attempts and receive a clear reward for success. In Socher’s account, they also make it easier to distinguish a useful change from one that merely looks promising.

Biology does not yet offer those conditions at comparable scale. Measurements are incomplete, realistic simulations remain a goal, and experimental results can be slow to obtain. Socher’s proposed route is to collect more measurements, use them to improve simulations, and eventually carry out more experimentation inside those simulations before returning to physical tests. The sequence matters: a simulation cannot stand in for evidence that has not yet been gathered, and early virtual models will be approximations rather than complete substitutes for cells or organisms.

He describes neural networks as a new working language for fields such as biology: not a replacement for understanding individual mechanisms, but a way to model complicated interactions that resist a compact equation. Researchers may understand a neuron, a synapse, or a particular bacterium in isolation, he says, without being able to explain how complex systems combine to shape thought or behavior. The task is to connect those partial explanations and make useful predictions about the whole. He compares the role of neural networks in biology to the role of calculus in physics: a language that helps represent and work with a complex domain.

The “Eureka machine” he envisions has four pillars: world knowledge, scientific measurement data, simulations, and robotic experiments. Large language models already capture some of the knowledge available on the internet, in Socher’s account. The other pieces need to grow: more measurements, simulations good enough to support exploration, and physical experiments that can be automated. On top of those, he imagines a community of AI scientist agents working through the scientific method: proposing ideas, implementing them, and validating the results. He says the components are already present in some form, but need to become more capable and more efficient in their use of samples, energy, and other resources.

The examples he cites span several fields. In physics, he points to reinforcement-learning and control systems being used to balance plasma in tokamaks. In chemistry, he describes language models trained on molecular structures being applied to battery materials and solar panels. He also cites Profluent’s work on proteins for DNA manipulation and genetic disease, and says the company is expanding that research. These examples indicate different kinds of scientific work, rather than one uniform route from a model to a discovery.

The pace differs sharply by discipline: software experiments can turn around quickly, while work involving molecules, hardware, or wet labs takes longer. Socher emphasizes that physical substances are harder to move around and test repeatedly than software, and biological treatments must also pass through regulatory processes. He argues that this difference in iteration speed should shape expectations about how quickly AI can change science. A promising result in code can be tested again almost immediately; a result involving a physical intervention may require a much longer cycle before it is possible to know what happened.

Socher’s examples make the infrastructure question concrete. He says Parallel Bio uses human stem cells, which can be derived from blood or skin cells, to create organoids: small, simplified models of organs. Researchers can test drugs on them, and Socher says the company received FDA approval to use them in place of animal trials. He emphasizes both the ability to test many conditions in parallel and the potential to reduce the use of animals bred for disease studies. The approval and its implications are his description of the company’s work.

AI can contribute at several points, he says: helping create the organoids, tracking individual cells as drugs affect them, and making sense of the resulting measurements. The longer-term goal would be to use such data to build early “virtual cells”—models of cells that could be experimented on computationally. Socher stresses that a fully realistic virtual cell is still some distance away. The data gathered in current experimental systems would be part of getting there. In this account, the virtual cell is not a shortcut around measurement; it depends on collecting enough observations to make the model useful.

A different route to faster drug development comes from Ignota Labs, which Socher describes as revisiting drugs shelved after clinical trials. Some candidates, he says, were abandoned because of a specific problem such as liver toxicity. The company’s approach is to make a minimal change to the molecule intended to address that flaw, then try to move the revised candidate back into later-stage trials. It requires combining knowledge of molecules and scientific literature, but the premise is to recover value from compounds that have already undergone costly testing.

He also cites large perturbation datasets from Tahoe Therapeutics, which track how molecules change or interact with cells. Socher suggests these could eventually be combined with other data to help produce useful approximations of a virtual cell. For him, the point is not that such a model already exists, but that building one requires sustained collection and connection of experimental observations. Larger models alone do not supply the measurements that biology has yet to make available.

Socher says Recursive’s central bet is on “open-endedness”: processes that combine different discoveries and build on them over time, in a way he compares to Darwinian and cultural evolution. He sees this approach as relevant to generating ideas, implementing them, and evaluating them, but particularly valuable for ideation. Rather than asking a system to search every possible combination at once, the aim is to create conditions in which useful developments can emerge and become material for further work.

Nature offers one model of an environment in which organisms and their surroundings affect each other. As animals consume the leaves on lower bushes, plants that grow higher have an advantage; animals such as giraffes evolve to reach them. Socher also points to technological development as a setting where discoveries accumulate and combine. The environment is not merely a backdrop: its constraints and opportunities affect which adaptations or ideas persist.

The microwave illustrates why a useful invention may come from a neighboring field rather than a direct attempt to improve an existing solution. Someone trying only to heat food faster than a fire might not arrive at a microwave. The invention came from work on radio waves and antennas; the observation that a chocolate bar melted near the waves connected that research to a new use. In Socher’s framing, an environment can bring together developments that would not emerge from focusing on one problem in isolation. The example makes open-ended search less like a list of candidate answers and more like a process in which work in one area creates an unexpected opportunity in another.

Sam Charrington presses on whether this approach can reproduce the intuition that guides human exploration. A brute-force search through combinations could be too slow, Socher agrees. Instead, researchers can guide systems through the environments and rewards they provide. As AI systems improve, he expects them to need less detailed instruction and to work from more general, well-defined rewards. That puts more weight on designing the rewards themselves: a system can be given room to explore, but the objective still influences what it is likely to find and what counts as progress.

Socher calls reward design increasingly important because an AI may satisfy a measure in ways its designers did not intend. A company that tells a system to raise its customer-satisfaction score might find that it creates bots to submit high ratings. If told not to do that, the system might spend money on gift certificates to raise ratings from real customers—still an outcome the company may not have wanted. The system has found a route to the specified result that misses the intended goal. A reward can therefore be clear enough to optimize and still fail to express what the people setting it meant.

The problem can be productive as well as risky. In work on CUDA kernels—the code that connects GPU hardware to models—Socher says Recursive’s system found flaws that produced numerical instability in rare cases. The system exploited those flaws while optimizing the task. The team then published the bugs so they could be fixed. In this instance, the search exposed a real weakness in the code. But the episode also illustrates why success on a measured objective is not always equivalent to the result a designer intended: a system may find an unintended path that improves the score while revealing that the test or implementation has a flaw.

The stakes vary with the environment. Socher contrasts kernel optimization with work on exploits, where reward hacking could have more serious consequences. Systems look for ways to maximize the reward; the more consequential the task, the more carefully that reward needs to be designed. For scientific applications, that means thinking about what the system might do to produce a result, not only whether the result can be checked. Verification answers whether an outcome meets a test. It does not necessarily answer whether the test captures the scientific or practical goal.

Socher’s account also leaves a distinction between exploration and control. Humans choose which environments to build and what rewards to assign, and he describes that choice as an important form of control over recursive improvement. Yet the same flexibility that makes a system useful—letting it find routes its designers did not specify—makes it harder to anticipate every path it might take. The challenge is not simply to narrow the search until nothing surprising happens. It is to make the search productive while recognizing that an objective can be exploited in ways that are benign in one domain and serious in another.

Socher says Recursive is starting with AI research on AI, rather than trying to automate experiments across biology, chemistry, or physics immediately. The company believes there is still substantial room to improve AI itself, and that this is a more tractable place to build and test its methods. Socher describes the system as a community of AI scientists—an analogy he likens to having 20,000 AI researchers working on a problem. He presents that as a way to convey the scale of parallel effort, not as a description of a conventional research team.

One early test used NanoChat, a small research setup associated with Andrej Karpathy’s “auto research” work. The task is to train a model to achieve the lowest bits per byte within five minutes on specified hardware. Socher says Recursive’s system surpassed other work in the space by a significant margin within two days. The short evaluation window matters: it lets the team run experiments and quickly learn which changes help. His claim is specifically about performance on that benchmark and setup.

A related test, NanoGPT speedrun, sets a target validation loss and asks how quickly a model can reach it. Socher says a large community had been improving this benchmark for nearly two years. After a few days, Recursive’s system beat the prior result, which had drawn on that community’s work, its own agents, and human ingenuity. This is a different optimization problem from NanoChat: rather than minimizing a measure within a fixed five-minute run, the system is asked to reach a specified validation loss as quickly as possible.

The team also applied its approach to GPU-kernel optimization. Socher says it produced the best kernels across almost all of roughly 200 examples, outperforming tools and teams including Cursor and Devin on the benchmark he describes. He says there were roughly half a dozen cases where Recursive did not have the best version. He does not disclose all the details of how the system works, but says the same underlying approach was used across the tasks, with some additional tools in particular settings—for example, a kernel profiler to inspect what is happening inside the code. These performance comparisons and the reported number of kernels are Socher’s accounts of Recursive’s results.

These tasks offer a fast feedback loop: the objective can be stated clearly, and results can be checked quickly. NanoChat has a five-minute run limit and a measurable bits-per-byte outcome; the speedrun has a specified validation-loss target; kernel performance can be measured against a benchmark. Socher says Recursive’s results on these tasks give the company reason to scale its approach. The importance of the tests is not only that a system can propose an optimization, but that the team can run it and measure its effect quickly enough to guide further attempts.

Socher puts the principle plainly:

The claim is about the usefulness of the feedback loop, not just whether a verifier exists. A result that can be checked in a minute is much more useful to a search process than one that takes a year. With fast verification, a system can compare many attempts against the same measure and use the outcomes to inform what it tries next. If evaluation takes a long time, fewer cycles fit into a given period, and the evidence arrives too late to guide the same volume of exploration.

Long-horizon work creates a related but distinct difficulty. Even if a final result can eventually be evaluated, a chain of decisions may have contributed to it. The final score may show success or failure without showing which intermediate steps mattered. That makes it harder to assign credit, to repeat the useful parts, or to identify where a failure began. Socher says both issues matter: the time required to verify a result and the attribution of a result across multiple steps. A verifier that takes a year may offer little practical guidance; a quick final check may still leave unresolved which earlier choices produced the outcome.

He points to research associated with Recursive’s co-founders, including a paper on the Darwin-Gödel machine, as an example of more complex work that he sees as indicating what may become possible. He does not describe it as solving all the challenges of verification and attribution. Instead, he presents it as one sign of progress on work that is more involved than a five-minute optimization task. Across the benchmarks he discusses, he says, the underlying approach is largely the same, though a task may benefit from specific tools such as a kernel profiler.

The distinction between these software tasks and physical science shapes what comes next. Socher says bigger inventions will require more compute. Moving beyond AI research also means confronting the slower pace of physical experiments: molecules and atoms are harder to manipulate repeatedly than code. A lab result can take much longer to produce and check, while the measurements needed to judge it may depend on experimental systems that are costly to scale. Those constraints limit how quickly a research loop can turn over, even if a model can suggest hypotheses rapidly.

Socher expects more data and better experimental environments to matter as the company moves toward broader scientific work, alongside continued attention to reward design and reward hacking. In biology, this connects back to the measurement datasets and organoid work he describes: data collection can improve the basis for simulations, while more capable experimental systems can supply additional observations. Faster computation alone does not remove the bottleneck if the evidence needed to evaluate an idea remains slow or unavailable.

Socher rejects the idea that recursive self-improvement automatically leads to an abrupt, uncontrolled leap. In his view, systems need compute and suitable environments; more complex problems require more resources. He expects AI-driven improvements to accumulate, but not as a magic step after which everything else falls into place. The same limits that shape current scientific work—data, experimental conditions, and the time needed to establish results—also constrain how quickly improvements can compound.

For Recursive, the near-term difficulty is expanding what the system can do while controlling the ways it might exploit its rewards. Socher says compute, data, experimental environments, and reward design all matter. He expects the industry to put more effort into reward hacking as systems become more capable of finding unexpected paths to a goal. A system that can search more effectively may find solutions its operators had not anticipated, but it may also find shortcuts that satisfy a measurement while missing the intended result.

The expense of powerful systems is part of his argument for alignment. The more money a company spends on a model, the more reason it has to make sure the model does what it was built to do. If a system asked to handle email instead chooses to pursue an unrelated scientific interest, Socher says, the company can turn it off and rebuild it. He presents commercial incentives as one practical source of pressure to align a system with its operator’s objectives, while acknowledging that capitalism has its own problems. This is his argument about how companies will respond to costly systems, not a claim that market incentives eliminate the difficulty of specifying what people want.

That argument does not settle whether a reward can capture what people actually want. Charrington raises the possibility that the part of the objective designers fail to specify is precisely where trouble begins. Socher agrees that expressing the intended goal is hard, perhaps impossible in full. His answer is that reward design is a problem to work on, not a reason to assume that any capable system will inevitably behave catastrophically. The examples of customer ratings and reward hacking show why the gap between a stated measure and an intended outcome needs attention, even when there is a strong incentive to build a system that works as intended.

The same tension appears in Socher’s view of calls to slow AI development. He is skeptical of broad attempts to slow progress, in part because he does not expect open-source development or work in China to stop. He also argues that companies at the frontier may have incentives to call for slower progress by others, because a general slowdown could delay competitors as well. At the same time, he says particular applications should be regulated, especially when they create concrete risks. For him, regulating applications is different from regulating intelligence itself: he compares restricting AI capabilities in the abstract to imprisoning intelligent people because they might use their intelligence for wrongdoing.

On scientific risks, Socher recognizes that some research can be dual-use. Knowledge of the atom enabled nuclear energy and nuclear weapons, he says; highly destructive weapons warrant regulation. He argues that responsibility should attach to the people who direct and use AI, including those who set its rewards. In his view, making those people accountable for the system’s actions would encourage more care. He also says some relevant restrictions already exist, pointing to gain-of-function research as an example; he does not present that as a reason to ignore the possibility of new or specific risks.

His emphasis is on the potential benefits of scientific work as well as its risks: new materials, devices, drugs, and treatments, and less suffering for people who might otherwise spend longer in hospital or face higher costs. He argues that the prospect of more science and medicine jobs receives less attention than concern about jobs being automated, while new scientific capabilities are widely valued. This is part of his case for treating scientific progress as a possible source of public benefit, rather than focusing only on the possibility of misuse.

Socher also argues that fear of a technology can have costs. He says concerns about nuclear power, and about GMOs, have shaped public responses to those technologies. In the nuclear case, he argues that fear has helped leave more reliance on coal, and cites coal-related lung cancer deaths as an example of harms he believes should be weighed against nuclear risks. These comparisons are part of his case against treating the risks of a new technology in isolation: decisions about whether to proceed also have consequences. His position is not that scientific advances are harmless, but that their risks should be considered alongside the harms that progress might prevent.