Multimodal AI
Models and products that work across text, image, audio, video, speech, documents, screens, and other modalities.
AI Compute Demand Is Pushing OpenAI Toward Google’s TPUs
John Coogan and Jordi Hays argue that growing AI compute demand is pushing companies to use a wider range of chips, cloud providers and data-center operators, even as Nvidia remains central. They cite reports that OpenAI plans to buy Google TPUs as evidence that compute needs can outweigh competitive history, while stressing that the market is still early and dependence on Nvidia remains substantial.
NVIDIA Links Road Tests, Simulation, and Vehicle Hardware in One Safety Loop
NVIDIA presents Halos SAFE as a framework for evaluating autonomous-vehicle safety by linking road tests, simulation, fleet data and vehicle hardware in a repeatable testing loop. Its central claim is that measured driving scenarios can be reconstructed with Omniverse NuRec, varied across conditions with Cosmos, and reused to test software changes and vehicle responses against safety targets. NVIDIA says the approach is intended to support evaluations from driver assistance through L4 robotaxis and provide evidence for certification across markets.
A Shopping Agent Revises Room Designs to Fit Budgets and Delivery Needs
OpenAI’s Vesper & Alder demo presents shopping as a series of connected decisions: a shopper can revise a reading-nook selection to meet a $250 budget without abandoning the design goal, then consider a throw as a separate add-on. The flow also lets the shopper correct the fulfillment choice, switching from store pickup to home delivery during the order process.
Apple’s Next HomeView Could Turn Its Screen Toward Whoever Is Speaking
Bloomberg’s Mark Gurman says Apple is already developing a higher-end HomeView with a robotic limb that would swivel the screen toward whoever is speaking. Unlike the $299 model, which users must reposition themselves, the proposed device would use visual and voice cues to identify a person and turn toward them. Gurman estimates it could cost more than $800 and arrive next year or the year after.
AI Is Linking Semiconductor Design, Fab Operations and Inspection
NVIDIA argues that accelerated computing and AI can help semiconductor manufacturers manage growing design and fab complexity by linking simulation, digital twins, engineering support, robotics and inspection. Its account centers on a feedback loop: simulation and production data inform models, while inspection results feed back into them. NVIDIA reports chamber simulations up to 35 times faster and more than 96% accuracy in wafer-map defect classification, figures it presents as examples of the approach rather than measures of overall fab performance.
At Extreme Compression, OCR Accuracy Can Hide Contextual Guessing
DeepSeek-OCR turns documents into compressed visual tokens and then reconstructs their text, a design DeepSeek presents as a way to process long documents with less context and computing cost. In his account of the paper, Two Minute Papers presenter Károly Zsolnai-Fehér highlights both the system’s strong OCR benchmark scores at high compression and a limit to what those scores prove: the decoder may infer missing words from context rather than recover them from the page. The paper also presents the pipeline as a way to generate training data at scale.
Private Document Processing Requires Controlling the Agent’s File Access
Filip Makraduli of Superlinked argues that sensitive document processing should happen before a coding agent sees the file: open models running on infrastructure the user controls can handle OCR, extraction and redaction, then return a smaller artifact for the agent to use. His Obsidian and MCP workflow is designed to reduce exposure to external model APIs and avoid spending context on raw documents, while letting teams choose or adapt models for specific tasks. But a private processing path does not alone guarantee privacy if the agent can read the original file first.
A Small Set of Choices Makes Model Decisions Nearly 10 Times Faster
OpenAI’s Decisions API is designed for applications that need a quick choice from a limited set of actions, rather than an open-ended response. In the launch demonstration, Romain Huet and Charlie Guo show how it turns text or image inputs into decisions an app can act on, such as routing sales leads or choosing a lane to steer a car. Huet says narrowing the model’s task to a few choices makes the API nearly 10 times faster.
Ecology Projects Combine Uploads, Real-Time Feeds, and Automated Model Analysis
The Sparrow Team at Microsoft’s AI for Good Lab presents Sparrow Studio as a workspace for bringing ecology data sources and analysis into a single project. Users can upload historical surveys or configure real-time ingestion from cameras and other devices, then assign models to analyze the data automatically. The tutorial’s central point is that each project ties together how data enters the system and how it is processed and managed.
Gemma 4 Brings Open Models to Browser and Mobile Devices
Paige Bailey of Google DeepMind argues that open models are becoming practical to run on ordinary hardware, including in a browser or on a phone, without sending data to a remote API. Her example is Gemma 4, a family ranging from 2 billion to 31 billion parameters that organizations can download, adapt and fine-tune; which version is useful depends on the device and its constraints.
Anthropic’s $518 Billion Compute Commitments Face Uncertain Customer Demand
Anthropic’s leaked financial figures show revenue rising from $400 million in 2024 to $4.6 billion in 2025, alongside a decade-long commitment to $518 billion in computing and infrastructure spending. On Big Technology Podcast, Alex Kantrowitz argues that those obligations could amplify the effects of slower growth, both for Anthropic’s suppliers and for a stock market increasingly reliant on AI-related companies. Ranjan Roy sees customer concentration and losses as more immediate risks, and expects Anthropic’s counterparties to have reasons to negotiate if the company falters.
Hybrid Filmmaking Brings AI-Assisted Production Back to Los Angeles
Luma AI COO Caroline Ingeborn argues that generative AI could help bring more film and television production to Los Angeles by lowering costs without replacing creative professionals. She points to “hybrid filmmaking,” in which actors and crews work onstage while AI places performances in different environments, as a way to make ambitious projects more feasible to produce locally.
Meta’s Consumer AI Strategy Puts Integration Ahead of Benchmark Leadership
John Coogan argues that Meta’s consumer-AI strategy is becoming clearer: rather than win every model benchmark, the company can pair capable agents with its infrastructure, hardware and reach. He sees Muse as a practical assistant for ordinary tasks, while Jordi Hays questions whether Meta’s devices will keep users beyond their initial appeal.
Task-Specific Detectors Can Beat Vision-Language Models at Runtime
Merve Noyan, a computer-vision and multimodal systems researcher at Hugging Face, argues that developers should stop deploying vision-language models as the default for vision tasks. They are too slow for many real-time uses, she says, and a task-specific detector can be faster and more accurate; licensing can also make popular model choices unsuitable. Her VibeVision toolkit instead gives coding agents task-specific models and a workflow that turns unlabeled images into a trained detector, with human checks on key decisions.
World Models Could Ease Robotics’ Training-Environment Bottleneck
World Labs CEO Fei-Fei Li argues that spatial intelligence—models that generate controllable, geometrically structured 3D environments—could extend AI beyond language and provide needed training and evaluation settings for robotics. She says the company’s Atlas model is designed to reconstruct and generate navigable worlds from images, while maintaining camera control and visual consistency. Li also argues that safety evaluation cannot rest with AI companies alone, calling for independent benchmarks and participation from academia, government, industry and civil society.
Studio 4.0 Keeps AI-Generated Video Editable on the Timeline
ElevenLabs presents Studio 4.0 as an agentic video editor built around an editable project timeline rather than one-off media prompts. The company says users can generate video, images, voiceovers, music and sound effects within the same workspace, then revise shots, captions and audio alongside the rest of the edit. Its Studio Agent can assemble a first draft or handle tasks such as captioning, while leaving editors able to alter the work and hand it back to the agent.
AI-Generated Video Enters the Editable Project Timeline
ElevenLabs is positioning Studio 4.0 as an agentic video-editing workspace where generated video, images, voice, music and sound effects remain editable within the same project. The company argues that users can generate and revise shots by prompt, then work with them on a conventional timeline through frame-level controls, captions, comments and dubbing. Its Studio Agent is presented as a co-editor that can create a first draft or carry out tasks such as captioning, while leaving the editor in control of the project.
Natural Voice Agents Must Balance Latency, Intelligence, and Cost
Boson AI co-founder and chief executive Alex Smola argues that voice agents will not become useful simply by producing accurate speech: they must respond, pause, handle interruptions and use tools at the pace and emotional register of human conversation. That makes latency, audio representation and inference cost product constraints rather than back-end engineering details, he says. As systems add vision and avatar-like presence, Smola contends, they will need to manage those trade-offs while learning when and how to adapt to individual users.
Full-Duplex Voice Agents Must Separate Conversation From Reasoning
Neil Zeghidour, co-founder and CEO of Gradium, argues that most real-time voice agents remain “walkie-talkies”: however quickly they respond, they can only listen or speak, not handle the overlapping signals that make human conversation work. Full-duplex speech models can represent simultaneous talk and backchanneling, he says, but they still sacrifice reasoning and tool-use capability relative to text-based agents. His proposed answer is a split architecture in which a lightweight speech model manages the live conversation while a separate text model handles harder reasoning and actions.
Claude Turns Product Links Into Editable AI Ad Workflows
ElevenLabs presents its MCP for Claude as a way to manage AI ad production through chat while retaining access to the underlying ElevenCreative Flows workflow. In a tutorial demonstrated by Alec Wilcock, users can start with a product link, approve stills and voice takes, revise motion or narration, and reformat a finished ad, while Flows exposes the separate image, video, voice and composition steps. The process still depends on user decisions—and on manual product details when a linked page cannot be retrieved.
MiniMax H3 Unifies Video, Image, and Audio References in One Generation
MiniMax argues that video generation’s separate tools for motion, subjects, frames and audio can be consolidated into H3, a model that takes text, image, video and audio references in a single generation. Presented by ElevenLabs in its ElevenCreative platform, H3 produces five- to 15-second clips with generated audio, timecoded multi-shot prompts and a 2K regeneration pass that retains the original instructions and references. H3 Max, built by fal from MiniMax’s published weights, trades that 2K option for a claimed roughly three-second generation time for a five-second 768p clip.
Parallel Agents Help GPT-6 Astra Test Theories and Solve DEF CON Puzzles
Ben Davis argues that GPT-6 Astra’s advantage on difficult DEF CON puzzles came from coordinated “sub-agent” workflows that test competing theories before an early mistake can derail a multi-step solution. After receiving the same official hints available to his group, Davis says Astra solved several puzzles they had missed—including one he says no one else had solved—by using parallel research branches to check visual evidence, operations and candidate interpretations.
GPT-6 Astra Adds Asynchronous Tool Calls and In-Flight Steering
OpenAI developer experience engineer Charlie Guo presents GPT-6 Astra as a model for longer, less-defined assignments that require it to retain context across tools, applications and changing user instructions. His central case is that asynchronous tool calling and in-flight steering in the Responses API let developers keep work moving—and redirect it—without cancelling a slow operation or rebuilding the task, alongside improved computer use and stronger creative, knowledge-work and 3D outputs.
Four Continuities Make the Fall-Through-Feed Effect Work
ElevenLabs’ tutorial argues that the “fall through the feed” effect is an editing illusion built on four continuities: a consistent subject and falling pose, scenes that begin empty and end with the subject exiting below frame, and identical upward motion for footage and social-media interface. Its workflow uses Nano Banana 2 and GPT Image 2 to establish reference and style frames, Seedance 2.0 to animate the falls, and Premiere Pro to synchronize the posts into a continuous scroll.