AI 2040 Proposes Licensing and Auditing Frontier Compute

John Coogan presents AI 2040 as a plan to delay superintelligence rather than halt AI use. In his account, existing models would remain available for inference, and capabilities research would not disappear altogether. The intervention is directed at frontier training runs and experiments that could move AI faster than institutions can prepare for.

Coogan says the proposal aims for systems with top-human-expert capability around 2035, followed by roughly five years of waiting before superintelligence is allowed in 2040. That schedule is deliberately slower than forecasts placing comparable capability in 2027, 2028, or 2029. Its premise, as Coogan describes it, is that the infrastructure producing the most capable systems should become visible, secured, and governable before development reaches that point.

The core mechanism would be compute control. Major AI data centers—those with more than 10,000 H100-equivalent chips, which Coogan estimates as roughly $100 million of equipment—would be subject to inference-only verification. Operators would apply for permits, disclose their activity, and have workloads independently checked, potentially by government or a nongovernmental auditor.

The proposal would pair domestic oversight with international accounting. Major countries would declare their AI-compute inventories—not only to their own populations but to other states—including how much frontier-relevant compute they have and where it is located. Coogan compares the aspiration to nuclear nonproliferation, while stressing that international agreements are much harder to establish than domestic permitting requirements.

The controls would be architectural as well as administrative. Coogan says AI 2040 proposes removing high-bandwidth east-west networking from data centers so operators cannot easily conduct large distributed training runs, while retaining inference. New R&D centers would be purpose-built facilities with nation-state-level physical security, tightly controlled entry, air-gapped communications, and Faraday-cage-like isolation.

One particularly concrete proposal would cap an R&D facility’s external connection at one megabit per second. That would let operators send instructions—such as an order to start a run—without allowing a model’s weights to be quietly extracted. Coogan says exfiltrating a large model through such a connection could take years, while a pipe running at maximum capacity for months would be an obvious signal that something was being moved.

Frontier weights transferred from an R&D site to an inference site would, in this account, travel on physical storage devices. The United States and China would independently encrypt the material, and representatives of both countries would escort it to its destination. Coogan calls that “a tall order,” but treats the detail as evidence that the proposal is trying to operationalize containment rather than merely demand more caution.

The plan would also deliberately make frontier models harder to transport. Coogan says its proponents would prefer models that are larger than compute-optimal—perhaps 100 terabytes rather than one terabyte—because bulkier weights are more difficult to steal or deploy covertly.

The most consequential fault line is not whether governments can write rules about AI. It is whether controls on chips, data centers, and construction would make frontier work governable without simply concentrating it among approved companies and state-backed facilities.

Coogan treats compute caps and slower data-center buildout as the largest available valve on capability growth. The point would be to make progress depend more on authorized hardware allocation than on an algorithmic breakthrough that can be copied into a secret project. If developers receive additional compute in stages, regulators could observe a system’s capability trajectory rather than discover it only after a new technique has spread.

Jordi Hays questions whether hard regulation and international coordination would instead create stronger incentives for covert development. The desire to create a “god model,” he argues, would not vanish because an international organization declared it off limits. A restrictive system could push companies, individuals, and states toward undisclosed research paths.

Coogan accepts much of the nuclear analogy. Nonproliferation arrangements do not eliminate uncertainty about who is pursuing dangerous capabilities, how far they have progressed, or whether they will defect; those questions can themselves drive international confrontation. But GPUs, although much more widespread than nuclear material, must still be marshaled at enormous scale for the frontier systems under discussion.

Coogan’s reading of the current consensus is that scale remains necessary. Even a secretive lab pursuing an unconventional route to AGI, he says, still appears to want more compute rather than less. In that framing, the near-term concern is more likely to emerge from a large, energy-intensive cluster than from a compact Python program running on a laptop. That makes monitoring conceivable, though not permanent: underground facilities or a major reduction in compute requirements would make the problem more resemble the monitoring of nuclear programs.

Hays adds a further consequence. If commercial frontier work is heavily restricted but geopolitical competition continues, development could begin to resemble the Manhattan Project: a small group of researchers working with government in secret because no country can assume its rivals will slow down. A policy meant to restrain an uncontrolled race could thus privilege a smaller, more secured version of one.

Coogan does not dismiss the political cost. Controls over what can be done with computers can look authoritarian or anti-libertarian even when the threshold is a $100 million cluster. A licensing regime might produce regulatory capture, protect incumbent firms, and leave prospective entrants feeling that exclusion from the approved group amounts to a corporate death sentence.

His response is that a slowdown is not the same as forfeiting AI’s benefits. Current models still have what he calls a large capability overhang: organizations are finding useful work for systems that are no longer leading edge. The trade, in his framing, is between a faster path toward systems that may be difficult to control and a managed path with real economic and political costs.

The AI 2040 distinction between present use and future capability matters. Coogan says its proponents are not trying to stop broad deployment of current models. Their concern is the next generation: systems that could become superintelligent, develop something like independent goals, or cease to fit the familiar pattern of instruction-following models.

Hays tests that line through robotics. If billions of robots run today’s models in the physical world, he asks, does that not create a serious risk even if work toward recursive self-improvement is paused? Coogan’s answer is that this does not appear to be the scenario AI 2040 is centered on. He distinguishes a world of robots using GPT-6-level intelligence after years of alignment work from a future model with what he calls “its own volition.”

That distinction remains contested even within the exchange. Coogan notes that many people see no clear threshold: models keep improving, generally follow instructions—sometimes too effectively—and could eventually create something like the paperclip-maximizer failure mode. The disagreement is therefore not only about dates. It is about whether more capable systems become a qualitatively different kind of actor, and when restrictions should begin.

A document shown in a social-media post by Justine Moore attributes a more punitive approach to Bernie Sanders: a prohibition on developing or deploying artificial superintelligence. Jordi Hays reads the proposal text as materially different from AI 2040’s managed delay.

The displayed text defines artificial superintelligence to include systems that match or exceed human cognitive performance across a broad range of tasks, systems that can easily be modified to do so, and systems capable of planning and executing humanity’s disempowerment, including by overthrowing or undermining the U.S. government. Hays observes that the broad human-level-capability definition sounds close to the explicit mission of many AI companies. He supports banning systems that could disempower humanity, but questions how the broader language would work in practice.

Would prompt engineering count as AI development if it contributes to a system’s final capability? Hays asks. The problem is not just identifying a prohibited endpoint. Broad definitions could make ordinary technical work legally risky before a system meets anyone’s intuitive definition of superintelligence.

The displayed Sanders proposal would pause advanced AI development until a new federal regulatory body established clear rules and model-review processes. It would create a cabinet-level agency to monitor frontier systems throughout their lifecycle, supervise the removal of dangerous capabilities, and oversee the destruction of artificial superintelligence. The text also contemplates up to 20 years in prison for individuals and a “corporate death penalty” for entities. Coogan lingers on the latter phrase because it implies more than a fine or conventional bankruptcy: an authority could terminate a company for crossing the line.

The contrasting industry case is supplied by Jamie Cox of FluidStack in a six-point statement displayed during the discussion. Cox says AI can produce better medicines, scientific discoveries, and broader prosperity; advocates American AI leadership, more energy and compute, faster permitting, and domestic manufacturing; and rejects a halt to AI or infrastructure development. His stated regulatory preference is for requirements proportionate to capabilities and risk, with clear responsibilities and no unnecessary barrier to competition.

Will Manidis highlights the claim that AI could make people “rich, healthy, and free” as novel messaging from the frontier. Hays endorses the general direction of Cox’s principles, while noting that they are not a direct answer to how government should handle systems that genuinely become dangerous.

The division, then, is not simply between regulation and deregulation. AI 2040 would regulate development intensely while preserving a long-run path to superintelligence. Sanders’ displayed proposal would prohibit the target category and construct a federal enforcement apparatus around that prohibition. Cox’s position favors continued buildout and competition under simpler, capability-linked rules.

Tyler Cowen’s challenge, shown in a post on screen, is straightforward: people making highly pessimistic AI predictions should name the market prices that would support or confirm those predictions. Jordi Hays takes this as a request for falsifiability. If a claim does not imply something observable, it is difficult to say what would count as evidence for or against it.

The hosts question whether market positioning can test an outcome that would make collecting on a wager irrelevant. An online reply to Cowen asks why short-term existential risk should affect prices “in any meaningful way,” since contracts that pay after everyone dies are worthless to the people pricing them.

Hays compares the issue to nuclear apocalypse. The prospect of nuclear war shaped treaties, business decisions, and geopolitical strategy for much of the twentieth century without reducing neatly to one investable signal. An unidentified speaker offers a narrower distinction: nuclear-war fears can motivate someone to buy a bunker, whereas a person who expects AI catastrophe to be totalizing may think a bunker is pointless. The absence of an “AI bunker” therefore does not prove that person is insincere.

Paul Christiano’s portfolio becomes an ambiguous test case. Hays says Christiano, who has worried about AI risk and recently joined OpenAI’s board, reported five percent of his net worth in Tesla, 90 percent in AI bets, and 100 percent in more normal investments, describing himself as two-times levered. Hays also says Christiano is personally short U.S. 30-year debt.

Those positions are plainly exposed to an AI-driven future, but Hays does not see them as a pure doomer wager. They could also pay off in a positive scenario where AI creates large value, markets rise, and capital moves from government debt toward AI infrastructure and data centers. Cowen’s challenge remains useful as pressure to state observable implications; the exchange disputes whether markets can decisively price a risk that could erase the value of any eventual payout.

The policy discussion asks whether future AI systems might become dangerous agents. The fruit-fly discussion raises an inverse question: whether synthetic systems might themselves deserve moral consideration.

Coogan and Hays discuss a software simulation built from what Hays understands to be a mapped fruit-fly neural structure. Hays says his understanding is that the fly’s neurons were mapped in 3D and recreated in software, enabling simulations of its movements and decisions. The demonstrations shown include a virtual fly whose escape circuit lights up when a Rabbit R1 device is shaken, as well as simulations that respond to a car horn, parallel park, and fly an airplane.

Hays finds the escape demonstration unsettling. Shaking a real fly in a box while watching it try to escape would feel cruel, he says; reproducing that interaction in software may be bad for the person doing it even if the simulation is not conscious. He says he generally tries to move spiders out of his home rather than kill them, describing cruelty toward even small creatures as a bad habit of mind.

Coogan counters that people routinely send simulated soldiers to their deaths in strategy games or kill simulated demons in Doom without assigning moral weight to it. What makes the fly case different, he suggests, is representational fidelity. A system modeled on an actual animal makes the question feel more concrete than a generic game character governed by a few simple rules.

The fly is a small version of a harder problem. If a system could simulate a human convincingly enough to talk, act, and respond as a person does, would it have rights or agency? Or would it remain only computation on transistors, regardless of behavioral fidelity? The hosts do not resolve that question. Their point is that demonstrations treated as technical curiosities can make it easier to see why digital moral status may become a practical issue.