Automation and Weaker Scaling Returns Could Widen Frontier AI Access

Sara Hooker argues that access to frontier AI may widen through two changes working together: automating the tacit work of model training, and shifting performance gains away from ever-larger pretraining runs.

The first is Adaption’s AutoScientist, which Hooker describes as a system for automating the full research loop behind training and alignment. Rather than giving users a fixed model and asking them to become better prompt engineers, it is meant to co-optimize data, training choices, and adaptation for a particular domain. The second is what she calls the “slow death of scaling”: the claim that pretraining size is no longer the most lucrative axis for improving models on the prevailing architecture.

Neither point means compute or large models cease to matter. In discussion, a questioner noted that smaller frontier models can still depend on larger models for distillation, and that post-training can remain GPU-intensive. Hooker agreed that large models remain useful and that distillation helps. Her narrower argument is that frontier labs are unlikely to quadruple pretraining scale again on the current architecture because the returns no longer justify it except perhaps for a small slice of the problem distribution.

That matters because pretraining compute is particularly centralizing. Hooker said it must be enormous, colocated, and provisioned for redundancy. By contrast, inference compute and other places where compute is applied can be more distributed. She pointed to post-training, test-time scaling, adaptive compute, and hardware co-design as areas where returns on compute are now higher than in pretraining.

AutoScientist is Hooker’s proposed mechanism for making that shift usable. Its value, in her account, is not simply automated hyperparameter tuning. It is reducing the specialized knowledge required to get a training run to work, increasing the chance that compute spent on customization produces a successful result, and allowing more questions to be explored.

Hooker frames the access problem as the product of a longer transformation in how science is done. Modern computer science, she noted, is only 77 years old—roughly two generations. But even within that short history, discovery moved from independent researchers to professional institutions, academic spaces, and industrial laboratories.

Professionalization brought benefits, but Hooker argues that it also produced what Rosanne Liu calls the “unreasonably narrow path.” To contribute to frontier AI, she said, a researcher has typically needed the right PhD program, advisor, internships, publication record, industry laboratory, and research problem at the right moment. Her own path included a PhD and work at DeepMind and other frontier labs.

The problem is not only that this is a difficult career sequence to acquire. It determines who is positioned to shape the technical questions at the frontier. Hooker estimates that fewer than 5,000 people worldwide know how to train frontier models at scale. Much of that expertise is not neatly captured in formal literature or curricula; it travels through accumulated decisions about configurations, data, architectures, and training runs, “as if” through an apprenticeship.

Compute compounded the filter. Hooker pointed to the growth of training-compute requirements for notable AI models and argued that industrialization shifted major AI contributions from academia toward industry. A map attributed to the AI Index Report and The Low-Resource Double Bind showed significant machine-learning systems from 2002 to 2022 as concentrated in a small number of countries, with large parts of the world absent.

The industry joke about being “GPU rich” or “GPU poor” captures a real division in her account. A handful of frontier labs build much of the technology used by everyone else. The resulting model is monolithic intelligence: the same broadly capable system is sent to billions of people, who are expected to become expert prompt engineers rather than gain control over how models are trained and adapted for their own work.

For Hooker, training expertise is therefore an exploitable search space. Humans carry experience and judgment, but they are not especially good at systematically navigating every configuration involved in training a model. Automating that work could make experimentation less dependent on an unusually narrow set of practitioners.

AutoScientist was released roughly four weeks before Hooker’s talk. She describes it as a system that self-improves across the training process, from data through alignment, selecting configurations based on a domain and its data.

Its search space includes epochs, learning rate, rank and alpha for LoRA, target layers, optimizer, warmup ratio, gradient clipping, weight decay, and other training choices. Its stated purpose is to optimize the whole loop rather than treating model training as a fixed procedure with a few settings adjusted by hand.

Hooker says the system outperformed Adaption’s research staff because its search is broader. Researchers bring useful experience, she said, but also priors shaped by particular model types. AutoScientist was tested across different model sizes, dense models, and mixture-of-experts architectures, and it can alter combinations of hyperparameters that people may be wary of changing simultaneously.

Adaption presented that comparison as a 35% relative improvement over human-configured training. Hooker’s interpretation was that a self-improving search procedure can exploit more of the available configuration space than a team operating primarily from accumulated intuitions. The practical objective is more predictable customization and less compute spent on unsuccessful runs.

But model-training choices alone were not enough. Hooker stressed that AutoScientist only produced the returns Adaption wanted once it also co-optimized data quality. She contrasted this approach with automated-research projects that treat data as something an agent may choose to create or modify. Adaption instead adapts the data systematically alongside the model.

That coupling of domain knowledge, dataset knowledge, and prior experimental knowledge is central to Hooker’s account. The aim is not merely to improve a general model’s default behavior. It is to adapt a model to an industry, language, specialization, or task while learning from prior work in related domains.

Adaption compared the original model with AutoScientist across business, finance, legal, marketing, medical, news, technology, and communication. The AutoScientist win rates were mostly above 60%, with legal at 59%. Hooker offered an important caveat about that clustering: the agent’s budget was configured to stop once it exceeded 60%. The results were a stopping condition rather than a performance ceiling. After the team removed that barrier, she said, performance continued to increase.

Demand in the beta was most acute in medicine, science, law, and code—areas where Hooker said general models often fall short and last-mile customization is particularly important. Adaption also reported a 34.3% relative quality improvement and a Grade A percentile of 57.7 in support of its claim that the approach reduces the innovation cycle.

Hooker’s ambition extends beyond choosing better training settings. She wants models to learn from their environment and adapt their behavior to the task rather than remain monolithic systems trained once, handed to a serving team, and used unchanged. She said Adaption chose to cover 242 languages from the outset and to work on both verifiable and non-verifiable tasks. The latter matters because much everyday work does not have an obvious correct answer or a simple automated grader.

The same logic applies at inference. Some tasks are difficult and some are easy, Hooker said; spending the same amount of compute on all of them is inefficient. Adaption intends to extend its work so that test-time compute also becomes adaptive to the task.

Asked about promising research directions, Hooker pointed to the balance between information stored in a model’s parameters and information held in nonparametric storage. That is a long-horizon optimization problem, she said. An “auto scientist for harnesses” would only work if the harness were co-optimized with the model, just as AutoScientist’s training results depended on co-optimizing data and model.

Hooker’s “slow death of scaling” claim rests partly on analyses of model performance over time. One tracked average scores for models below 13B parameters on the Open LLM Leaderboard, showing that models of the same size had become more capable. Another compared the best daily submitted model under 13B parameters with models above 13B, highlighting larger models that underperformed the leading smaller model.

Her conclusion is not that small models reliably beat large ones. It is that smaller models increasingly outperform many larger models, and that recent increases in size have not produced the same stepwise improvements associated with earlier generations.

Hooker attributes this to architecture saturation. Pretraining scale had been a powerful lever, but she argues that the prevailing architecture has a ceiling. A new architecture could have a different ceiling; architecture determines the limit. On the current one, however, she does not expect a frontier lab to quadruple model size again for pretraining.

The relevant question is marginal return, not whether capacity matters at all. Hooker said organizations are moving post-training further back into development because it is becoming a larger lever than size alone. Data quality can help use capacity better, and distillation remains valuable. Supersizing a model may still be worthwhile for part of the long tail, but she does not see it as a broadly compelling strategy.

That changes the competitive terrain in her view. As optimization expands from algorithms into data, interaction, and task-specific behavior, training recipes, domain knowledge, and experimental systems matter more. Compute remains an advantage, but it does not necessarily operate in the same winner-take-all way as an era dominated by increasingly large, colocated pretraining runs.

A questioner raised the safety objection directly: if models can be adapted more readily, what prevents people from adapting them for harmful purposes?

Hooker did not deny the risk. Making a tool more widely available creates a risk profile, she said, and she does not consider open source risk-free. But she distinguished AutoScientist’s stated role from the separate question of whether frontier models should be released openly. AutoScientist is intended to give organizations more control over customization, whether locally, privately, or inside their own companies.

In the broader debate, Hooker sees two facts that are often collapsed into a binary. Wider access can create genuine risks, while concentrated access limits who can participate in building and shaping AI. The balance is delicate, she said, and absolute positions on either side miss important parts of the problem.