The 1% Rule for Finding Durable AI Opportunities

Jeff Dean offers a practical test for founders deciding whether an AI product has room to persist alongside rapidly improving general models: begin by testing the best available model on the actual workflow.

The result matters less as a benchmark score than as a diagnosis. If a general model can already handle meaningful portions of the task—Dean uses an example of roughly 20% success—that may be a warning sign. It suggests the underlying capability is already appearing, and that more training data, larger models, or further scale may materially improve performance within six or 12 months.

More promising territory, Dean says, is where the model “succeeds zero percent or one percent of the time.”

That is not a guarantee of a business or a durable technical lead. It is the beginning of a sequence of questions.

First, test whether the failure is real. Is the model unable to perform the task, or does it merely need a better interface, prompt, or workflow? A task that the model partly solves may be close enough to its existing capabilities that general improvement will close the gap quickly.

Second, identify what changes the result. Dean points to several possibilities: access to information the general model cannot see, a focused model trained for a specific domain, or a system that can evaluate and improve results in a way the base model cannot on its own.

Third, assess the time horizon. Google’s general models are intended to become capable across an increasingly broad range of work. Founders should therefore ask whether their advantage is likely to disappear in six months, 12 months, two years, or perhaps only after a fundamentally different product, data source, or specialized model exists. The answer should affect the choice of problem.

Dean’s first filter is more basic still: work on something the founders genuinely care about and believe would be useful. A team that is unenthused by its own problem, or pursuing something that will not matter much even if it works, begins at a disadvantage. The question is not simply whether an AI capability is technically possible. It is whether solving it would make a positive difference in the world.

He sketches two concrete ways a small team can have an advantage despite the resources and breadth of a frontier lab.

One is access to data that a general model does not have. A product helping someone organize their own personal information, for example, can operate over material unavailable to a general model. Google may be organized around the world’s information, Hu notes, but personal information remains a distinct problem. The startup’s advantage is not necessarily a better base model; it is visibility into the information needed to solve a user’s actual problem.

The other is a highly accurate specialized model in a narrow domain. Dean points to AlphaFold as the shape of the opportunity: a system built for protein folding rather than a general-purpose assistant that happens to know something about proteins. Comparable approaches may be possible in materials science, chip design, and other fields where the right training data and a focused objective can produce an affordable but unusually capable model.

The 2001 Google search system and the TPU project both began with a change in how a bottleneck was framed. Hu recounts that Dean and Sanjay Ghemawat recognized that Google’s search index could fit in the aggregate RAM available across the machines running search. Moving from hard drives to memory, in her account, made search dramatically faster, and the system was shipped in a few days.

The TPU began with a different kind of back-of-the-envelope calculation. Around 2013, deep-learning-based speech recognition was producing much better results than Google’s previous speech systems. Dean says the error rate was cut in half—an advance he likens to 20 years of prior progress compressed into a few months of model and data work. But the new models were computationally expensive. If users began dictating emails and talking to their phones for a few minutes a day, Google’s server fleet would need to grow substantially just to support speech.

The response was not to accept CPUs as the fixed substrate. Google built hardware optimized for the computation machine learning actually needed: low-precision dense linear algebra. A TPU could not run Chrome or Word, Dean notes, but its narrow specialization made it useful for inference. The first system, developed over the following years, was 30 to 80 times more energy efficient than CPUs and GPUs of that period, he says, with latency 20 to 30 times lower.

The principle was not to make the chip maximally narrow. Google knew machine-learning algorithms were still changing, so it built what Dean calls a general-purpose linear-algebra system: specialized enough to support large multiplier units, high-speed memory, and eventually high-speed interconnects among many chips, but broad enough to accommodate evolving algorithms.

Dean sees a similar opening now in inference. Training can tolerate latency in ways interactive products cannot. For agents to become widely available, inference needs to be both low-latency and energy-efficient. He asks the audience to imagine what becomes possible if latency falls by 50 times.

The physical economics matter because moving data can cost far more energy than calculating on it. Hu cites a ratio Dean confirms: a multiplication may cost roughly one picojoule, while bringing data from high-bandwidth memory on an accelerator into the processor can cost roughly 1,000 times as much. That gap is not a low-level implementation detail. It shapes the algorithms and product behavior built on top of the hardware.

Batching is the familiar consequence. Rather than moving data for one example or token at a time, systems process many items together and amortize the energy cost of bringing data into the processor. In training, large batches are standard because they improve hardware efficiency; in a hypothetical world without this movement-versus-compute gap, batch-size-one training would be more plausible. But batching is a poor fit for low-latency inference, which is why Dean is particularly interested in inference-specific designs.

His current hardware questions are correspondingly concrete: minimize data movement; use extremely low-precision operations where possible; avoid supporting many precision formats if the required formats are known; and design around the actual workloads rather than a broad menu of general-purpose computation. The premise is that many apparent “model problems” are really energy and I/O problems in disguise.

Diana Hu frames the shift as a move from “better models” toward everything surrounding a model: retrieval, tools, memory, and agent workflows. Dean agrees that a model is one part of an overall system intended to solve a problem.

Context matters because it is directly available to the model for the task at hand. By contrast, the material in a model’s training data has been mixed into an immense parameter space—“trillions of tokens stirred together into a soup,” as Dean puts it. Retrieved documents, relevant history, tool outputs, and explicit instructions are clearer inputs. They can tell a model what matters now rather than requiring it to recover the right knowledge from training.

The difficult work is orchestration. A capable system needs to understand which tools are available, choose which will help at the next stage, break a problem into a sequence of calls, try multiple approaches, and evaluate whether those approaches worked. Dean expects the coordination of complex agents and multi-agent systems to become increasingly important.

For builders, the practical route into context engineeringThe design of retrieved information, tools, instructions, and memory supplied to a model for a task. is not abstract study. It is using models, harnesses, and tools on real tasks; observing where they fail; and changing the setup around them. A founder generally cannot alter the parameters of a closed model, but can write better guidelines, supply relevant information, and define skills that explain how to use tools for a particular class of work.

Dean offers an internal example from performance engineering. He and Ghemawat had been improving low-level libraries using a micro-benchmark framework. The normal human loop is familiar: measure existing performance; change code; rerun benchmarks; expand the benchmark suite; inspect cache footprint; and iterate. Since the relevant data structures might run across millions of Google processes, small improvements can matter.

They wrote a skill that taught an agent how to execute this loop in appropriate sequences: measure performance, propose code changes, test the result, and continue improving. The contribution was not a different model. It was the translation of an experienced engineer’s process into a form the model could use.

Dean says Google published a 30-page document, Performance Hints, that he and Ghemawat wrote about performance techniques. Others have summarized the material for models and found that doing so improves the models’ ability to reason about code-performance issues. The broader point is that a well-made skill can make a base model more useful without retraining it.

Jeff Dean believes many people still underestimate how long agents may eventually operate. Rather than working for an hour or two, systems with sufficiently capable models may be able to run for days or weeks on complex tasks. One example is translating an entire software system into a different programming language to gain better safety or performance properties.

But the present limitation is plain: agents often fail after 10 interactions with tools, or drift badly by step 30 or 50. Dean attributes much of this to distributional limits. Models work well on tasks they have enough experience doing. Move them off that path, and performance degrades; move farther away, and failure becomes more likely.

The reliability problem is therefore not solved merely by letting one agent continue longer. Dean’s alternative is to search. Multiple agents can pursue different approaches; another model or agent can evaluate which trajectories appear promising; the system can retain useful paths and discard the ones that have gone off the rails. He describes this as using inference-time compute to search across plausible solutions, improving both performance and reliability in long-running workflows.

At Google, Dean says, internal coding agents use skills for code review, performance measurement, retrieving log files, and other proprietary workflows. The underlying model was not necessarily trained on Google’s particular way of accessing logs. But a clear definition of how a tool works can make the tool usable, expanding what the agent can do without changing its underlying weights.

Clear specifications are the corresponding human responsibility. A person handing an ambiguous task to another engineer may rely on shared context and follow-up questions. An agent may instead infer a different goal, constraints, or definition of completion. As agents take on more of the execution, Dean says, the importance of specifying what is wanted has gone up.

Software translation is a case where agents already perform especially well because the specification is unusually complete. A Python implementation states what a Go implementation should do. Tests can be translated and rerun; behavioral discrepancies can be compared until they disappear. The model is not being asked vaguely to “build something useful.” It has a detailed behavioral contract.

For a founder managing dozens or hundreds of agents, the implication is that design documents are not administrative overhead. They are the mechanism by which a human’s intended outcome, rules, and constraints reach a larger amount of AI-assisted execution.

If the mechanics of building become cheap, Dean argues, the high-level choice of what to build becomes more consequential. The scarce resource is not simply the ability to execute a research plan or produce working code. It is taste: deciding which problem is worth the attention of a powerful fleet of agents.

That is how Dean frames research as well. A researcher may possess techniques and tools, but most of the battle lies in selecting the problem. Solving a well-chosen problem is more valuable than executing a polished investigation of a boring one. He does not expect models to be especially strong at this kind of judgment on their own; people will continue to steer large amounts of AI-assisted computation.

Taste is difficult precisely because it often lacks a clean measurable objective. Experience helps: working on many problems teaches a person which combinations of prior techniques and unresolved issues might yield something useful. Dean proposes a more deliberate practice as well. Write down a number of things that may matter over the next 12 months, choose one to pursue if appropriate, then revisit the list a year later. Which predictions mattered? Which did someone else build? Which did not materialize? The exercise creates more samples for calibrating judgment.

He also recommends thought experiments that challenge assumptions others treat as fixed. One such exercise begins with the chip industry’s longstanding pursuit of nearly error-free, deterministic transistors. At the scale of distributed systems, engineers already use unreliable components to create reliable systems: data is replicated across machines and racks, and techniques such as Reed-Solomon encoding preserve it through failures.

What if that logic were pushed down toward the transistor level? Dean imagines a device built from transistors that might produce 20 errors a day instead of one error every million years. A system designed for that world might use redundant signaling paths and radically different fabrication or design methods. He is explicit that this is not a recommendation to build such a system; decades of existing practice may rest on good reasons. But revisiting a foundational assumption can expose a new design space.

Two earlier systems came from this form of reframing. The TPU asked whether hardware could be specialized for machine learning before machine learning had become as central as it is now. MapReduce emerged from a different observation: Google’s crawling and indexing code was full of hand-built parallelization, checkpointing, and reliability machinery, obscuring the relatively simple computation many jobs were actually trying to perform.

Dean and colleagues recognized that many of those computations could fit a MapReduce abstraction. The simple map-and-reduce logic could sit above a lower-level system responsible for checkpointing and robustness. The result made large-scale computation more reliable and reusable, not by eliminating difficult systems work but by moving it beneath an interface.

Jeff Dean predicts more automation of machine-learning development itself. The pattern is an automated scientific method: break a high-level objective into subproblems, run experiments on those subproblems, evaluate the results, combine the useful outcomes, and produce an improved system.

The idea applies beyond ML. Any domain with a measurable objective can potentially benefit, he says, from a tight loop of proposing, implementing, evaluating, and revising. The limiting factor is often not whether an evaluator exists but how quickly it can produce an answer.

Dean gives an example from quantum chemistry. Researchers may want to generate a molecular configuration and evaluate its properties using a density functional theory simulator. But a single high-fidelity simulation can take a night of computation. His colleagues trained a neural approximation on inputs and outputs from those expensive simulations. The approximation was nearly as accurate, he says, but 300,000 times faster.

That changes the research process. Screening 10 million candidate configurations need not be a six-month computational campaign; it might happen over lunch. Faster evaluation permits more experiments, which permits broader search, which makes the entire discovery loop more productive.

The same structure could apply to model design. Today, a large research team generates ideas, tests them at small scale, promotes the promising ones to larger-scale experiments, and incorporates the results into a new recipe. Dean sees no fundamental obstacle to automating much more of that process, with humans perhaps providing high-level nudges about directions worth exploring. The objective becomes discoveries per unit of compute input.

This is the condition behind Dean’s idea of self-improving ML systems: not a system that advances without measurement, but one that can run many experiments against meaningful evaluators at low latency. Chip design, science, engineering, and machine learning itself become more tractable when the loop can be automated and accelerated.