Frontier AI Pacing Proposal Pits Safety Oversight Against Competition

“Pacing the frontier” has moved beyond a general appeal for caution in AI development. As John Coogan described Dario’s proposal, it is a three-part institutional program: give independent third-party evaluators access to AI companies; let frontier labs in democratic countries establish safety standards together with government support; and seek international coordination on AI pacing, including with authoritarian governments such as China.

The first step would put outside evaluators in a position to verify whether labs are following their safety practices. The second would address a legal obstacle to formal collaboration: companies that agree together to constrain development, deployment, or other competitive activity can face antitrust concerns. The third is intended to address the strategic problem that a unilateral slowdown may simply leave the field to countries that do not participate.

Coogan traced the public emergence of the idea to a July 28 open letter associated with employees and leaders at OpenAI, Anthropic, DeepMind, Meta, and other organizations. He said Sam Altman publicly suggested that AI development might need to be paced that day. Coogan also said OpenAI subsequently told Axios that it had helped shape the petition’s language and that Altman had discussed pacing with White House officials.

1. July 28
   An open letter associated with employees and leaders at major AI organizations calls for “Pacing the Frontier”; Coogan says Sam Altman also raises the prospect of pacing AI development.
2. August 18
   Coogan says OpenAI publishes “Pacing Model Development in an Era of Cybercritical Capabilities” and reports slowing scaling and pausing a major reinforcement-learning run.
3. August 31
   Coogan says Anthropic publishes a statement explicitly discussing pacing the frontier and coordinated pacing mechanisms.
4. September 12
   Dario publishes “We Must Pace the Frontier,” which Coogan describes as turning the idea into a larger public Anthropic campaign.

Coogan said Elon Musk endorsed Dario’s essay shortly after publication, with Altman following. He said Demis Hassabis generally agreed with Dario’s concerns while arguing that the proposal needed work. The immediate argument was not over whether a lab can choose to be cautious. It was over whether safety requires a standing apparatus of evaluators, coordinated standards, and government-sanctioned cooperation.

Jordi Hays relayed Gavin Baker’s characterization of the proposal as broader still: embedded third-party evaluators; a national regulatory regime for models beyond a capability or ingredient threshold; stricter limits on compute distillation for China; coordination among democracies; and a separate international arrangement encompassing China. In Hays’s account, Baker understood the Sherman Act waiver as a prerequisite for formal coordination among Anthropic, OpenAI, and other frontier labs.

The evaluator proposal is the plan’s most concrete element, but it immediately raises the question of who would perform the review. Coogan said Dario named METR as an example, and that the choice attracted scrutiny because people have moved between METR, Anthropic, and OpenAI.

That overlap does not settle the question for Coogan. People deeply embedded in AI-safety work may have the expertise and seriousness needed to assess frontier systems. But it is not the same arrangement as a conventional accounting firm reviewing a company in an industry it has no stake in. The evaluators, in this model, would be participants in the safety discourse and potentially connected through staff or investors to the labs under review.

David Sacks made the skeptical case in a post Coogan read aloud. Sacks described Anthropic and OpenAI as holding a frontier-intelligence duopoly by market share, revenue growth, and model capability. If unreleased systems are concerning enough to warrant slower progress, he argued, those companies can slow down voluntarily.

His objection was not to responsible conduct by labs. It was to the claim that responsibility requires permission from Washington, a suspension of antitrust constraints, or an evaluator network that he said was intertwined with the companies’ personnel and investors. Coogan summarized Sacks’s position as: if the models are genuinely alarming, the companies should act on that judgment themselves rather than construct a regulatory framework around it.

The scope of any evaluator regime matters as much as the evaluators’ identity. Coogan worried that a compliance structure created for the largest model developers could spread outward. He used the example of an “Instinct-like” small team building an AI product without training a frontier model. If such a company had to spend a year negotiating with evaluators, host embedded reviewers, and wait for government decisions, he argued, it could make a breakout consumer product much harder to build.

Hays said he had not seen a comparable push to regulate application-layer companies. He referenced Greg’s view that someone training a model at home for personal use should not face those limits. The proposed target, as the hosts described it, is frontier development. But the debate turns partly on whether a system built for frontier labs can remain confined there.

The second plank—frontier labs jointly setting safety standards with government support—is economically and legally different from executives agreeing that safety matters. Coogan’s point was that formal cooperation among competitors can run into ordinary antitrust rules.

He used airlines as the analogy. If American Airlines, Delta, and United agreed that there should be fewer flights “for safety reasons,” the effect could be less supply, higher prices, stronger margins, and worse outcomes for consumers. The Sherman Antitrust Act exists to restrain this kind of concerted behavior among powerful firms.

That does not mean AI labs cannot individually delay a training run, limit a deployment, or adopt their own safety practices. The complication arises if they seek binding common standards that govern development or access across competitors. In Coogan’s account, that is why Dario calls for government involvement: a formal arrangement may need explicit approval or an exemption to avoid being treated as collusion.

Sacks’s critique makes the same distinction into an accusation. The leading labs, he argued, should not present a voluntary decision to slow down as something that requires collective permission. They should act independently if they believe the danger warrants it.

Baker’s case, as Coogan relayed it, pointed to a different reason for third-party review. The tangible new development, Baker said, was that OpenAI and Anthropic would have embedded evaluators from still-unknown organizations. Baker considered that sensible because model outputs do not have a Section 230-style liability shield. A documented duty of care, he argued, could matter in future litigation; he noted that some internet companies might have faced bankruptcy without Section 230 protections.

The disagreement is therefore over the mechanism of accountability. One side sees pre-deployment evaluation and coordinated standards as necessary guardrails. The other sees product liability and voluntary restraint as sufficient incentives—and sees a formal compact among the leaders as a risk of cartel-like control.

Dario’s proposal to coordinate with authoritarian governments takes the issue beyond domestic governance. Coogan framed the difficulty in terms of equilibrium: countries may agree that rapid capability gains create risks, while each still wants to preserve an advantage over rivals.

He compared the problem to nuclear nonproliferation. A country that wants to stay ahead cannot easily offer an arrangement that asks a competitor to accept permanently inferior capabilities. Coogan said the world instead became multipolar, with multiple countries pursuing meaningful nuclear capacity. His question was whether AI coordination could avoid the same strategic logic.

Donald Trump presented the opposing emphasis in posts shown on screen. Trump wrote that the only AI “guardrails” needed were a “STRONG and SMART (High IQ!) PRESIDENT,” said his administration already had substantial criminal and regulatory power over AI companies, and argued that opposition to AI and data centers benefited China.

Trump also criticized Dario directly, saying the administration had stopped AI figures from doing “bad, or potentially bad,” things and accusing him of posing as a “perfect little angel.” Hays found the suggestion that leading labs were involved in a conspiracy against AI incongruous; Coogan said it did not follow.

In another post read by Hays, Trump treated calls for regulation as suspect precisely because they came from industry leaders. He asked when leaders of an industry had ever sought rules that, if strongly implemented, could drive them “into oblivion and bankruptcy.” He called claims that AI could destroy humanity a hoax, said China would not impede AI, criticized U.S. permitting constraints, and described AI and data centers as an economic-development engine larger than oil, gold, diamonds, or the internet.

Coogan’s concern about international pacing and Trump’s argument for building both start with competition. They differ on whether managing catastrophic risk requires coordination that can survive geopolitical rivalry, or whether the attempt itself sacrifices U.S. advantage.

A central criticism of the proposal is that safety rules would protect Anthropic and OpenAI by raising compliance costs for challengers. Hays argued that the inference is too easy. A policy can be good for a company and good for safety at the same time; that overlap does not establish that the stated safety rationale is cynical.

Baker, as Hays summarized him, called Dario’s the weekend’s most maximalist proposal and said it would probably benefit Anthropic over the long run. But Baker also said he believed Dario was sincere.

Hays offered a competing commercial scenario, relaying Roon’s view that pacing could compress rather than protect frontier-lab margins. If OpenAI and Anthropic stop extending their capability lead, Google DeepMind, xAI, and other competitors could close the gap. A market with more comparably capable providers could bring sharper price competition.

Hays cited R. Karaszi’s observation that AI spending fell somewhat in August largely because of price cutting, not because customers had stopped using AI. In the hosts’ discussion, the immediate competitive pressure was not necessarily open-source Chinese models. It came from established providers competing for enterprise demand.

The reason, Hays suggested, is operational rather than purely technical. Companies may be more comfortable buying from OpenAI, Anthropic, Google, AWS, or Azure because they expect reliable service, uptime, and support. Downloading open weights may be cheaper or technically viable, but it can create operational burdens that an enterprise would rather avoid.

The hosts did not resolve which commercial scenario is more likely. Regulation could increase entry costs, as critics warn. A pause at the frontier could also allow other well-resourced competitors to catch up, weakening a lead and putting margins under pressure. Coogan said Altman had told Fortune that OpenAI had long told investors it was not guided solely by conventional shareholder interests. As a public benefit corporation, Coogan suggested, it could argue to investors that slower growth or lower margins are acceptable if leadership believes safety requires it.

Coogan raised a supply-chain question: if AI-safety advocates want to restrict chip production or frontier-compute buildout, should electricity generation be paced as well?

His answer was no. Chips have a relatively direct relationship to advanced-compute capacity, while electricity is a general industrial input. Even if frontier AI development were constrained, Coogan argued, capital and engineering talent could still go into nuclear power, solar, wind, batteries, grid reliability, and the broader U.S. energy supply chain.

The argument is also a hedge. If society eventually concludes that AGI or ASI cannot safely be built, Coogan said, it would still benefit from abundant cheap energy. Greater generation could ease grid tension and energy costs without committing society to unrestricted frontier-model scaling.

Jordi Hays agreed that the distinction broadly made sense but emphasized how diffuse power is as an input. Chips can support many uses, but their relationship to data-center capacity is comparatively narrow. Electricity can be used for nearly everything.

Coogan pushed the point to an extreme: would safety regulation eventually target a South Korean company that makes materials used in semiconductor production because constraining it might slow AI progress? The exchange did not establish a clear stopping point. It did identify the problem with deep supply-chain controls: the farther policy moves from frontier models toward general economic inputs, the harder it is to distinguish AI governance from broad industrial constraint.

At the All-In Summit, Jensen Huang put Trump on speakerphone during an onstage appearance. Coogan said Huang thanked Trump for social-media posts that had pushed back on AI alarmism. Hays said Huang had been anti-doom throughout the current AI cycle.

The call did not produce a new policy framework. Its significance, in the hosts’ telling, was alignment: Trump’s insistence that the United States should keep expanding AI and data-center capacity has support from a prominent leader building the chips and infrastructure on which that expansion depends.