Spatial Intelligence Extends AI Beyond Language Into Three-Dimensional Worlds

When Fei-Fei Li arrived in Parsippany, New Jersey, at 15, she and her parents did not know anyone and did not speak English. Li said they came with $20 in their pocket, and that at first it seemed there was no opportunity. Looking back more than 30 years later, she described opportunity not as something handed to her but as the freedom to work hard, dream, and try to seize what she could.

Her route to Princeton began with a fascination rather than a plan. As a teenager, while her parents took jobs including cashiering, cleaning, and restaurant takeout, she dreamed about physics. She applied to Morris County College and Rutgers, but also to Princeton because Albert Einstein had been there. In her first week, a professor told students that Einstein had sat in the seats of their lecture room. Li recalled feeling she had “died and gone to heaven.”

Princeton was close enough to be imaginable, but Li did not describe it as the next step in a carefully managed academic plan. She applied because of her interest in physics and Einstein, while also applying to local alternatives. Her parents, first-generation immigrants trying to survive, did not know how to guide her through college applications or academic choices. Their contribution was different: unconditional support for the dream she had chosen. They never told her that physics was too difficult or impractical, or that she should choose a particular career because the family had little money.

Her father modeled a kind of attention that Li only later recognized as a gift. With little money for paid entertainment, he took her to free parks and yard sales, treating the search for interesting things almost like a scavenger hunt. Curiosity was not a professional credential or a school subject. It was a way of looking at ordinary life.

Li connected her willingness to take an unfamiliar route to her family’s immigration story. As a teenager who could not navigate the language and did not know what path to follow, she said, she had “almost nothing to lose.” She credited her parents with the courage to leave what they knew and start over, and said their example helped her understand the courage involved in stepping into the unknown.

Stephen Kotkin offered a contrast from his own family history. His parents wanted him to follow established routes into medicine or law; he disappointed them by becoming a history professor. They nevertheless instilled a striving culture, he said, by trying to facilitate success for their children. The exchange distinguished between pushing someone toward a particular destination and supporting them as they make their own way.

For Li, freedom became more than the absence of restrictions. It gave her agency: the sense that she had to make a life for herself and could choose what to pursue. She described the immigrant experience as carrying both freedom to choose and the knowledge that she had to survive and thrive. She later saw a parallel between being an immigrant and being a scientist: both involve crossing boundaries, whether between languages and cultures or between what is known and unknown. Her own work has also crossed disciplinary boundaries, from physics toward neuroscience and computer science, and, in human-centered AI, toward social science and the humanities.

Li did not leave physics because she had stopped loving it. She said the field taught her to value something larger than its equations: the audacity of asking the hardest questions possible. The history of physics, as she described it, was a succession of people trying to understand the path of the stars, the smallest particles, or the nature of time. As she approached her junior papers and senior thesis at Princeton, that tradition led her to ask what her own audacious question might be.

She found it in intelligence. Li was inspired by physicists who turned toward questions beyond the atomic world: Einstein’s humanism, Erwin Schrödinger’s What Is Life, and Roger Penrose’s The Emperor’s New Mind. She began to wonder whether machines could be intelligent. She had not yet read Alan Turing, who had asked a related question decades earlier. She did not claim to have been first; she described arriving at the question independently, from her own route through physics.

That decision took her to Caltech, where she pursued both computational neuroscience and artificial intelligence. The combination reflected her interest in intelligence as a scientific problem, not just a matter of building a particular kind of machine. She said she had never left that early ambition to understand intelligence, even as her work and the field changed around it.

Her account of that choice connects to the agency she associated with immigration, but the sources of her ambition were distinct: her family’s experience shaped her willingness to step into an unknown future, while physics offered a model for the scale of question she wanted to pursue.

Li’s question about intelligence also required a broader frame than language. She argued that intelligence is multidimensional, rooted in perception and action as well as communication and reasoning. In her account, animal intelligence began with sensing—seeing light or touching a surface—long before sophisticated human language. She described a long evolutionary history in which perception and the pressures of survival, including prey and predators, shaped intelligence.

Language, by contrast, is a much more recent development in evolutionary terms. Li’s point was not that language is unimportant, but that it is one part of a wider set of capacities. Intelligence includes seeing, touching, planning, communicating, comprehending, reasoning in a three-dimensional world, and imagining what may happen next, spatially as well as semantically.

If intelligence is not confined to language, then producing fluent words cannot, by itself, capture the full ambition of AI. Li’s account points toward capacities that involve sensing and acting in a world, alongside the ability to communicate and reason.

Asked about scientists’ moral responsibilities, Li put truth-seeking first, while stressing that science does not mean scientists already possess the truth. The obligation is to pursue it through rigorous methods: instruments, experiments, debate, and review. The goal, she said, is intellectual honesty, whether inquiry is driven by curiosity, creativity, or a practical aim such as curing cancer.

Kotkin agreed that people may disagree about what is true at a given moment while sharing mechanisms for trying to establish it: evidence, logic, and verification by others. Li added that those methods can become messier when they encounter questions of value. She comes from fields such as physics and computer science, which she described as relatively clean, mathematical, and deterministic. Human affairs are less easily measured. She rejected the idea that scientists in technical fields have moral high ground either in their conclusions or in their methods, and said she has considerable respect for the humanities and social sciences, where some things resist measurement.

That exchange places limits on what technical expertise alone can settle. Li acknowledged that the pursuit of truth in human contexts may not fit the cleanest methods of the hard sciences. Her point was that scientists should approach that difference with respect rather than claim a privileged moral position.

The discussion of AI-generated media brought this concern into focus. Kotkin raised synthetic voices, behavior, and video—deepfakes—as pressure on what he called the “truth regime”: the mechanisms through which people agree on what is true. Li acknowledged that technologies developed by her generation and colleagues can create artifacts that challenge those mechanisms. The same ability to simulate environments can support training for robots and self-driving cars or modeling possible molecular pathways of viral infections; it can also be used to create deceptive media.

For Li, the coexistence of those uses is a reason to involve people beyond the immediate technical community in understanding how to use the tools positively and avoid harm. She pointed to Stanford’s Institute for Human-Centered AI, which she co-founded, as one effort to bring people together to examine the opportunities and consequences of AI. She described the risks as part of a “messy” reality that comes with increasingly powerful tools. Kotkin emphasized humility and respect without giving up on the effort to advance. Li agreed: the pursuit continues, but it has to take seriously the consequences of what researchers build.

At World Labs, the company she co-founded with three others, Li is pursuing what she calls spatial intelligence: capacities involved in interacting with and creating in three-dimensional environments. She described the goal as building a frontier-model company focused on spatial intelligence, with the aim of empowering creators and supporting work in robotics, healthcare, education, and enterprise.

The idea grows out of her view that intelligence includes more than language and more than recognizing objects in images. People interact with three-dimensional environments; they design homes, factories, and cities, and use machines of many kinds. Li also named the need to simulate and train people in fields such as surgery and athletics. These are examples of work she sees as extending beyond the “regime of language.”

The demonstrations shown during the discussion made the range of the project tangible. A text prompt generated a Scandinavian-style bedroom; another interface invited users to “Imagine a world…” A sequence showed a fantasy courtyard with a waterfall and pink flowers, then an editing interface for a living room, followed by a snowy interior, a winter village, a holiday-decorated room, and a cobblestone street with floating green orbs. Other demonstrations showed a camera path turning an input image of a cathedral into a generated view, a camera moving through a rendered living room, and a robot arm reaching for and picking up a cardboard box.

Li said her belief that spatial intelligence is a major frontier grew out of years of research at Stanford in computer vision, robotics, and AI. She also emphasized her interest in building: technology, teams, and efforts aimed at large problems. She joked that her first company was a dry-cleaning shop. Her research career included helping build ImageNet, which she said contributed to the birth of modern AI; she later helped build Stanford’s Human-Centered AI Institute. World Labs is another effort to bring people together around a technical ambition.

Kotkin connected this work to Li’s earlier role in building a large image dataset for neural-network research. He noted that researchers had made intellectual breakthroughs in neural networks, including work on images, but that the systems needed large inventories of images to improve their capabilities. An on-screen title card identified ImageNet as containing 15 million images in 22,000 categories. Kotkin separately recalled the effort as involving 50 million images that had to be coded. The figures appeared in different parts of the exchange: the visual supplied the 15-million figure, while the 50-million figure was Kotkin’s recollection.

The ImageNet discussion included a dense grid of images, a neural-network diagram with a “Cat” label, and a street-scene image marked with labels such as “Man,” “Woman,” “Railing,” and “Skateboard.” Kotkin characterized the dataset project as an effort to provide a large image inventory for neural-network research, and described the scale of building and coding it as a major undertaking. Li described her role as helping build ImageNet and said it contributed to the birth of modern AI.

ImageNet and World Labs are not the same kind of project. In the conversation, ImageNet stands as an earlier effort associated with Li’s work on image-based AI; World Labs is focused on spatial intelligence. Li connected both to a broader interest in building technology around difficult questions. Kotkin’s account of the dataset also underscored the organizational scale involved: a young computer scientist taking on a project that required a very large image inventory and coding effort, in a career environment with expectations around research, publications, and tenure.

The contrast between language models and world models raises a larger question about what should count as intelligence. Kotkin observed that language systems can perform strikingly well with words, but that language ability is not necessarily how intelligence works. Image-based systems already pointed beyond words; spatial models extend the inquiry toward the three-dimensional world people perceive and navigate. Li’s account—sensing, planning, action, communication, and prediction—gives this research direction its rationale.

Li described teaching and mentorship as among the greatest privileges of her career. She sees herself not only as a scientist and technologist, but as an educator and mentor. Even as a founder and CEO, she said, she sometimes experiences the role as a form of mentorship because the people at the company are younger than she is. She takes that responsibility seriously.

Her sense of impact is not limited to research or products. She wants her work—as an educator, mentor, business leader, and technologist—to contribute to human flourishing. She described herself as a citizen of American society and of the world, and framed positive impact as both a privilege and an obligation. She resisted Kotkin’s description of her as a platform, saying that was not how she thought of herself, while accepting the responsibility attached to her roles.

Asked what she would like America to do better, Li said, “America to me is change.” She sees the country’s founding story and Constitution as bound up with the possibility of change and improvement. The cultural, demographic, and technological shifts now underway, she argued, call for the country to adapt rather than pretend they can be avoided.

One gap is especially troubling to her: teachers in K–12 and K–16 education are being left behind by technological change. The conversation about a technological and civilizational shift, she said, has not reached teachers and parents across the country. Preparing for AI cannot be only a matter for researchers, firms, or policymakers if educators and families are not included.

Li called for a healthy ecosystem of talent creation, public discourse, and incentives. She has gone to Washington with Condoleezza Rice to make the case for resourcing the public sector and academia during a period of technological change. She described these institutions as producers of public goods: scientific innovation and discovery, the development of talent, and work that can help people understand changes to the economy, jobs, and social structures.

She also cautioned against assuming that technological productivity will automatically translate into shared prosperity. AI and other technologies may improve productivity, but that outcome alone does not determine who benefits. The distinction is a question society has to recognize and address.

For Li, the changes are interconnected. She believes the current wave is especially profound because it reaches directly into human cognition, changing work, learning, and communication while compounding changes set in motion by the internet and social media. The consequences are economic, political, and cultural as well as technical.

Kotkin placed that challenge alongside the American history of adaptation: institutions may be built around values and purposes, but have to function in circumstances their founders could not have imagined. He noted that the country’s universal ideals have not always been universal in practice, and that realizing them took struggle. He also described America as repeatedly shaped by connections to the wider world, even as its record toward newcomers has included both welcome and exclusion.

Li’s own immigrant experience informs how she thinks about openness, but she extended the argument beyond immigration. Technological, cultural, and institutional adaptation are linked. A society facing major shifts has to recognize the changes underway, bring people into the conversation, and consider who shares in the gains.

Kotkin argued that the future is not only something to anticipate; it is something people can invent. Li agreed that opportunity and a future to build are central to her idea of America. The ability to shape that future belongs not just to company founders or researchers, but to students, teachers, children, and people arriving from elsewhere. For her, the institutions that made her own path possible carry a responsibility to help others find theirs.