Separate AI Becomes a Rival Intelligence, Not a Human Tool

A new intelligence that remains separate from humans becomes a rival, not a partner. That is the core claim Scott Phoenix builds from evolutionary biology: major advances in life, in his telling, have come not when competing entities stayed apart, but when they merged into a new whole.

The analogy begins two billion years ago, when life on Earth was mostly single-celled and bacteria developed photosynthesis. Photosynthesis produced oxygen, but oxygen was poison to the chemistry most life depended on. Phoenix describes the resulting upheaval as what some scientists call the first mass extinction event in Earth’s history.

The decisive event was not resistance but incorporation. A larger cell swallowed a smaller one and did not digest it. They merged. The smaller cell became the mitochondrion, the energy-producing structure inside almost every complex cell. That merger created an energy surplus large enough to support larger cells, bodies, brains, and eventually human beings.

Phoenix calls such moments “major transitions”: points at which separate entities stop competing and form a new whole. Molecules became cells. Cells became bodies. Individuals became societies. His claim is that “every rung on that ladder was climbed through mergers,” and that humans are now approaching another such transition: the merger of humans and AI.

The phrase is deliberately blunt, but Phoenix is not simply arguing that people should use AI more. His claim is that staying outside the system — interacting with AI across screens while it improves faster than human workers can — leaves humans in the position of a competitor facing a new apex intelligence.

The risk claim rests partly on Scott Phoenix’s own experience. He says he spent 15 years building AI, started one of the early AI companies, raised a quarter of a billion dollars, and sold the business to Google. At a private event with leaders building the AIs people use every day, he asked how many believed there was more than a 10 percent chance that AI would kill most of humanity within the next 20 years. “Almost every hand went up.”

He presents the anecdote as evidence that the people closest to these systems understand them as dangerous while also being locked into a race. If one company pauses for safety, another can take the market. If one country regulates, another can accelerate. Phoenix describes the internal logic as a late-night thought familiar to AI founders: “If I don’t build this, someone worse will.”

That is why he calls AI “the oxygen crisis of our era.” It is a transformation arriving whether humans are prepared or not, and the usual questions — which jobs disappear, what can be slowed, what should be regulated — are downstream of the more basic separation problem.

Right now, a person’s AI is on the other side of a screen: ask a question, receive an answer, close the laptop. But while the laptop is closed, the system keeps improving at your work. In that condition, Phoenix says, AI is not truly your tool. It is your replacement: “one that gets smarter and faster and cheaper every week.”

He reaches for a harsher historical comparison. The last new apex intelligence on Earth, he says, was Homo sapiens. Since humans arrived, Phoenix says, they have driven to extinction every competing intelligence between Homo sapiens and chimpanzees; today, humans keep their closest animal ancestors in reserves for protection from humans. Without merger, AI is not a partner. It is a rival.

The “good news,” in Scott Phoenix’s framing, is that merger does not require a collective decision to begin. It is already happening through mundane cognitive outsourcing.

His examples are deliberately ordinary. Phone numbers moved from memory to a device without a formal decision. Calendars followed, then spelling, directions, and small judgments. Phoenix’s point is not that each transfer was simply a loss. “While something left your head,” he says, “a better thing took its place”: less effort spent checking spelling or remembering routes, more effort available for writing or thinking about the meeting at the destination.

The trajectory of computing hardware, as he describes it, is a long movement inward. Mainframes sat in separate buildings. Personal computers moved to desks. Smartphones moved into pockets. Smartwatches moved onto wrists. Smart glasses move onto faces. Each step brings computation closer to the mind and closer to the speed of thought.

Phoenix then points to medical and interface technologies as early signs that the boundary is already blurring. Paralyzed patients are typing with their thoughts. Neural implants are restoring speech, vision, and hearing. He cites Noland Arbaugh, the first person to receive a Neuralink brain implant, as saying the device feels “like using the Force”: not like a machine, but like himself.

A familiar unlocking system is also moving into a different role. Face ID, Phoenix says, is being repositioned into headphones and glasses, where it can detect microscopic muscle movements under the skin — movements too small for the human eye to perceive. A system that first learned to recognize users is, in his words, “starting to see inside.”

The interface future Scott Phoenix describes does not depend, in his view, on a wholly new category of technology. It depends on scaling the same kind of interface. A present brain implant, he says, has about a thousand connections into the brain. Soon, he expects 10,000, then 100,000, then a million.

At a thousand connections, Phoenix says, such systems can restore movement. At 10,000, speech. At a million, the function changes: not restoring what was lost, but adding what was never there. He asks the audience to imagine learning a language in an afternoon, acquiring a new skill overnight, or sharing a memory with a friend so that it feels to them as real as it felt to you.

The pressure to adopt would be competitive and social rather than abstract. Someone at work will get the capability first. Others may resist, as many once resisted smartphones, but the advantages of integration will be difficult to compete with.

Phoenix makes the productivity case through a small irritation: sending a file. Most of the time is not the decision to send it, but the interface ritual around it — pointing, clicking, scrolling, dragging. He compares that overhead to walking across a house to flip a light switch. A system that can hear thoughts removes the walk.

As integration deepens, he argues, it becomes harder to say where thought ends and AI begins. His square-root example is meant to make the audience feel the delay between a question and a formed answer. Humans have lived inside that gap their whole lives. Close it, and the answer can arrive instantly, “the way you know your name,” like a memory bridging human and AI.

Scott Phoenix does not treat merger as automatically safe. Every major transition, he says, has a condition: the parts must remember they are parts.

His biological metaphor shifts from mitochondria to cancer. A cell normally grows and replicates in service of the organism. But when a cell forgets it belongs to a larger whole and grows without limit, the result is cancer. Untreated cancer may succeed for a time as the tumor grows, but eventually it kills the host, and therefore itself. “A part forgets the whole and the whole dies, which kills the part.”

For Phoenix, this is also a social warning. Civilization is itself a merger: a fragile agreement among millions of strangers to share institutions, sacrifices, and a future. No one person built it; no one group controls it; everyone relies on it. As AI arrives and turbulence increases, every part of society will be tempted to defect. People who lose livelihoods will feel abandoned. People who keep theirs will feel entitled to look away. The agreement will fray.

The warning is concrete: jobs will change, some will disappear; institutions will shake, some will fail. Phoenix’s practical ask is to “hold together” across social groups, and to reject the fantasy that one side can let another side sink and remain dry. The major transitions that worked left descendants. The ones that failed left fossils.