Data and Training
Training data, synthetic data, fine-tuning, post-training, reinforcement learning, data pipelines, labeling, and dataset quality.
Pretraining and Attention Infrastructure Made Vision Transformers Practical
Isaac Robinson of Roboflow argues that transformers overtook convolutional networks in vision not because images stopped needing visual structure, but because that structure moved from hand-built architecture into pretraining, scaling and tooling. In his account, ViT-style models first lacked the inductive biases and efficiency that made CNNs dominant, but self-supervised vision pretraining and attention infrastructure from the LLM world made the simpler architecture practical. Robinson frames the next problem as deployment: turning large foundation backbones into model families that can meet real latency, cost and hardware constraints.
Data Scarcity, Not Compute, Is the Next AI Bottleneck
At AI Ascent 2026, Flapping Airplanes co-founders Ben and Asher Spector argued that data scarcity, more than compute alone, will determine where AI can create value next. They said the biggest gains so far have come in unusually data-rich domains such as search and coding, while much of the economy — including robotics, trading, science and narrow industrial workflows — lacks comparable datasets. Their proposed answer is to make models far more data-efficient by developing new GPU-level primitives that current frameworks such as PyTorch make hard to express.
Voice Will Be the Primary Interface for AI Agents and Robots
At Sequoia’s AI Ascent 2026, ElevenLabs co-founder and CEO Mati Staniszewski argues that audio was an overlooked frontier in 2022 because the AI field was focused on text and images, leaving room for a smaller company to build quickly and monetize early. His broader case is that as AI intelligence becomes more capable, voice becomes the interface problem: the way people will use agents, robots, services, education and healthcare. Staniszewski says the next hard problems are emotional intelligence, timing, authentication and workflow, not merely making synthetic speech sound human.
Luma Is Rebuilding Video AI Around a Unified Multimodal Transformer
In a Stanford CS153 guest lecture, Luma AI co-founder and chief executive Amit Jain argues that generative video is only a staging point toward “unified intelligence”: models that understand and generate across text, images, video, audio, code and tools in a single work loop. Jain traces Luma’s path from Apple-era LiDAR and 3D capture to internet-scale video, saying the company followed the data but now sees prettier clips as insufficient. The destination, he says, is a multimodal AI factory for professional creative and physical work, where human skills, tool use, feedback and unified transformer architectures produce full campaigns, schematics, productions and eventually robotics workflows.
Descript Bets Creator AI on Reliable Editing, Not Content Slop
Laura Burkhauser, Descript’s chief executive, distinguishes generative AI tools for creators from the “slop” she defines as mass-produced content arbitrage. Her case is that Descript’s future depends less on adding AI everywhere than on making editing automation reliable, reversible and useful for recorded human media. That means choosing third-party models by fit and taste, building in-house systems where Descript has workflow data, and treating creator backlash as a product constraint rather than a branding problem.