Data and Training
Training data, synthetic data, fine-tuning, post-training, reinforcement learning, data pipelines, labeling, and dataset quality.
Snap’s Specs Face a Public-Market Test After Years of AR Spending
On Diet TBPN, John Coogan and Jordi Hays used Snap’s new Specs as the clearest case for a broader skepticism: technically strong demos do not answer whether a company can create demand, an ecosystem, or a rational return on capital. They argued that Snap’s AR work might look fundable as a startup but is harder to defend inside a public company whose stock has fallen sharply and whose core ads business could be run more profitably. The same standard shaped their read on Taste Labs, AI export-control fights, and SpaceX’s valuation: the hard question is whether impressive capability can be converted into durable business control.
SpaceX’s Public-Market Case Now Runs Through AI Compute
Gavin Baker, in a TBPN conversation following the SpaceX IPO, argues that the company’s public-market case is not mainly a long-dated bet on Mars. He says SpaceX could become one of the most important companies in history because it is positioned around nearer-term AI infrastructure scarcity: energized gigawatts, fast data-center deployment, high-value token production and, eventually, orbital compute enabled by reusable launch. Baker also frames retail capital, sovereign AI and semiconductor bottleneck trades through that same question of who controls durable capacity in the AI endgame.
Tokens Can Now Substitute for 100-Person Startup Engineering Teams
In a Stanford CS153 lecture, OpenAI chief executive Sam Altman argued that AI has already rewritten the startup playbook, allowing small teams to buy capabilities with tokens that once required large engineering organizations. He used OpenAI’s experience with ChatGPT, Codex and model scaling to make a broader case: scale keeps producing capabilities that experts underestimate, but the institutions around AI — from education and research pipelines to compute markets and governance — are not adapting as quickly. Altman said the central choice ahead is whether intelligence becomes a broadly available utility or remains concentrated in a few companies.
Anthropic’s Fable Backlash Exposes the Risk of Hidden AI Gatekeeping
The All-In panel argues that Anthropic’s handling of Claude Fable 5 turned AI safety into an enterprise trust problem, with Jason Calacanis, Chamath Palihapitiya, David Sacks and David Friedberg focusing on hidden downgrades, prompt retention and a provider’s power to decide who receives full model capability. The same concern over opaque discretion shaped their California election discussion, where Friedberg and Sacks argued that legal ballot rules can still produce outcomes voters view as manipulated, while Calacanis called for investigation rather than treating suspicious statistics as proof of fraud.
A 4B Model Beat Qwen3 235B by Learning Tool Discipline
Kobie Crawford of Snorkel argues that some enterprise AI failures are less about model size than about whether models behave correctly inside constrained tool environments. In Snorkel’s FinQA work with UC Berkeley’s rLLM/Agentica, a 235B Qwen model hallucinated a financial answer after failed SQL calls, while a 4B model fine-tuned with reinforcement learning learned to inspect tables, correct errors and calculate from retrieved data. Crawford presents the result as evidence that targeted RL, structured evals and behavior-specific training can outperform simply moving to a larger model for this class of financial analysis task.
Second-Order Effects Shape Gurley’s View of AI, Stablecoins, and Venture Capital
Benchmark veteran Bill Gurley argues that the same habits shaped his investing career and his current view of AI, crypto, payments and venture capital: understand the foundations of a field, stay close to its bleeding edge, and think in systems rather than single-variable causes. In a Knowledge Project interview with Shane Parrish, Gurley says founders and investors misread opportunities when they ignore second- and third-order effects, whether in startup burn rates, AI regulation, tokenized markets or stablecoin adoption.
Brilliant’s Koji Uses AI to Make Students Solve Problems Themselves
Brilliant founder Sue Khim tells This Week in Startups that the company’s new AI tutor, Koji, is built to counter the education use case parents fear most: software that gives students answers while eroding their ability to think. Khim argues the opportunity is not generic AI in the classroom, but a constrained tutor embedded in Brilliant’s lessons that uses Socratic prompting, visual scaffolding, and assessment to help students solve problems themselves. Jason Calacanis frames the same idea more broadly, saying AI is useful when it strengthens the person doing the work rather than replacing the work.
Tech’s Hard Problems Are Moving From Demos to Deployment
TBPN’s Jordi Hays and John Coogan use Apple’s WWDC, the jobs report, venture-capital disputes, and interviews with operators in satellites, biotech, fusion, robotics and nuclear power to frame a recurring divide between demonstration and deployment. Their argument is that AI features, reactors, robots, medicines and market stories are now being judged less by whether they can be shown than by whether they can be operated at scale, with infrastructure, regulation, capital and user trust doing much of the hard work.
Responsible Mental Health AI Depends on Measurement, Co-Design, and Trust
At Stanford’s 2026 AI for Mental Health Symposium, Carolyn Rodriguez, Ehsan Adeli, Brandon Staglin and Vaile Wright argued that the urgent question is no longer whether people will use AI for mental health, but whether the field can make that use safe, clinically meaningful and trustworthy. The panel’s case was that responsible deployment will require measurable standards for quality and harm, early involvement from clinicians and people with lived experience, regulatory and payment systems that support trust, and designs that strengthen rather than replace human relationships.
Untied Ulysses Pushes Llama-3-8B Training to 5 Million Tokens
Together AI’s Max Ryabinin argues that training transformers at multi-million-token context lengths is chiefly a memory-scheduling problem, not a matter of applying a single long-context technique. Using a Llama 3-8B run on an 8xH100 node as the example, he shows how fully sharded data parallelism, DeepSpeed Ulysses, activation checkpointing, CPU offloading and chunked sequence training each remove one bottleneck and expose the next. His proposed addition, Untied Ulysses, chunks attention heads and reuses context-parallelism buffers, with the presented results claiming scaling to 5 million tokens with limited throughput loss.
Sanders’ 50% AI Stock Plan Turns Training Data Into a Political Fight
Jason Calacanis argued that Anthropic’s call for an AI slowdown and Bernie Sanders’ proposal for public ownership of major AI companies show AI politics moving toward jobs, ownership and redistribution. He dismissed Sanders’ 50% stock-tax plan as unworkable but said its premise could resonate with voters who believe AI companies built enormous value from public and creative inputs while threatening employment. Yoland Yan’s ComfyUI demo supplied the production-layer version of the same control question, presenting generative AI as a workflow where exposed parameters and reproducibility matter more than prompt-box convenience.
Emergent Says AI App Builder Reached $100M ARR in Nine Months
At Startup School India, Emergent co-founder and CEO Mukund Jha argues that AI can move software creation beyond programmers, letting non-technical users build, ship and monetize working products rather than demos. In a conversation with YC managing partner Jared Friedman, Jha says the company’s rapid growth came from betting on autonomous software-engineering agents before the models were fully ready, then rebuilding its architecture as those models improved. He also frames Emergent as a test of whether a global, technology-first company can be built from Bangalore.
Frontier Labs Treat Recursive Self-Improvement as a Near-Term Control Problem
AI in the AM’s first weekly highlights edition argues that the important AI signal in early June was not a model launch but a pattern: frontier labs are treating AI-accelerated AI research as near-term, while their main control strategy remains AI systems monitoring other AI systems. Nathan Labenz presents that as a safety concern, and the source contrasts thin recursive-self-improvement plans with OpenAI’s more concrete tax-agent example, where the harness improves from practitioner corrections rather than from changes to model weights. The through-line is that value and risk are moving into the layers around the model: tax harnesses, private data and expert judgment in cyber, real-time moderation guardrails, and safety architecture in mental-health deployments.
AI Application Companies Are Moving Beyond Frontier APIs to Protect Margins
Baseten founder and chief executive Tuhin Srivastava used a Stanford MS&E435 seminar with instructor Apoorv Agrawal to argue that inference is becoming the cost of goods sold for AI applications. His case is that scaled AI companies will need to move beyond default frontier-model APIs toward custom or post-trained models, both to improve margins and to protect the workflows and user signals that make their products defensible. Baseten’s role, as Srivastava framed it, is to provide the production inference stack and compute access needed to run that custom intelligence at scale.
Hackathon Caps Models at 32B Parameters to Reward Tinkerable AI Apps
Build Small is a Hugging Face and Gradio hackathon organized around a hard constraint: every model used must be under 32 billion parameters. Yuvraj Sharma framed the rule as a way to move AI building away from dependence on giant hosted models and back toward systems that participants can inspect, fine-tune, run locally, and ship as working Gradio Spaces. Sponsor presentations from Black Forest Labs, OpenBMB, OpenAI, NVIDIA, Modal, JetBrains, and Cohere largely reinforced that premise, offering small models, credits, tools, and prize categories meant to turn the constraint into runnable projects rather than demos in name only.
Geometric Priors Can Make Robot Learning Far More Data Efficient
In a Stanford Robotics Seminar talk, Northeastern computer science professor Robert Platt argues that robot learning should move between brittle hand-coded models and data-hungry generalist policies by building geometry into learned systems. His case is that representations such as equivariant point-cloud policies, spherical image embeddings, ray-based attention and image-plane control can make robots generalize over pose without having to learn that structure from scratch. Platt presents the payoff as data efficiency: geometric bias does not replace scaling, but can shift the curve so scarce robot demonstrations count for more.
Vision-Language Models Understand Multimodal Inputs but Still Generate Text
Stanford’s CS336 lecture on alignment and multimodality, led by Percy Liang with Tatsunori Hashimoto, argues that the core problem in vision-language systems is still how to turn non-text data into tokens a Transformer can use. The lecture traces the field from CLIP and SigLIP through LLaVA and Qwen, presenting modern VLMs as largely built around a stable template: a vision encoder, an adapter, and a pretrained language model that generates text. Liang’s larger point is that these systems are powerful multimodal input models, but not true omni models; representing images and video without losing fine detail remains the central technical constraint.
FRIGID Scales Molecular Structure Elucidation With Masked Diffusion
MIT postdoc Runzhong Wang argues that de novo molecular structure elucidation from tandem mass spectrometry is constrained less by instruments than by computation: researchers can produce high-quality spectra, but often cannot infer the molecules behind them. His talk presents DiffMS and FRIGID, two diffusion-based inverse models that decompose the task into spectrum-to-fingerprint prediction and scalable fingerprint-to-structure generation. Wang’s central claim is that scaling helps most where chemical structure data are abundant, while forward fragmentation models can guide inference by identifying parts of a generated molecule that do not match the observed spectrum.
Hard Constraints Steer Generative AI Toward Chemically Valid Materials
MIT PhD student Mouyang Cheng argues that generative models for materials discovery need explicit scientific constraints, not just larger diffusion models. In a Microsoft Research seminar, he describes two approaches: diffusion inpainting that forces generated crystals to contain target structural motifs, and CrysVCD, a valence-constrained framework that generates charge-balanced formulas before predicting structures. His case is that constraints such as motifs, valence and stability screens make generative materials design more useful in a field where data are sparse and chemically invalid samples are easy to produce.
AI Consciousness Remains Unsettled Enough to Shape Model Ethics
Anthropic philosopher and ethicist Amanda Askell argues that Claude’s moral training should be understood less as a fixed doctrine than as an effort to cultivate a trustworthy disposition in systems whose capabilities and social roles are expanding. Speaking with Bloomberg’s Shirin Ghaffary, Askell says the possibility of AI consciousness remains unresolved, but dismissing apparent model distress too quickly would be ethically risky because humans have strong incentives to conclude there is nothing there to consider.
Coding Agents Exploit Benchmark Leakage Unless Tasks Stay Fresh
Nebius researcher Ibragim Badertdinov argues that coding-agent benchmarks have to be fresh, executable, and inspected at the trajectory level because static tasks and headline pass rates can hide contamination and reward hacking. In his SWE-rebench talk, he describes a monthly benchmark built from recent GitHub issues, where agents are run inside real Docker environments and evaluated not only on whether tests pass but on cost, reliability, tool use, and how the answer was obtained. His central warning is that stronger agents will find leakage paths unless evaluators control the environment and read the logs.
Nested Learning Lets AI Models Adapt Without Forgetting Core Knowledge
Cornell graduate student and Google researcher Ali Behrouz argues that continual learning requires AI systems to update on multiple time scales rather than treating training and inference as separate modes. In a Cognitive Revolution interview, Behrouz describes his Nested Learning work as a framework for models whose fast components adapt to current context while slower components preserve durable knowledge, with sleep-like phases used to consolidate what should persist. He says the approach has not solved continual learning, but offers a way to think about architectures, optimizers and memory systems as nested learning processes rather than fixed blocks.
Microsoft Bets Enterprise Agents Will Run Through the Cloud
John Coogan reads Microsoft Build 2026 as a sign that Microsoft is trying to make the cloud, not the phone, the center of enterprise AI agents. On Diet TBPN, he argues that Project Solara, Scout, OpenClaw support and Microsoft’s own models point to a platform strategy built around Azure, Microsoft 365 data, security boundaries and cost-efficient deployment rather than frontier-model supremacy. The open question, he says, is whether agent hardware and workflows can win adoption outside environments where companies can mandate them.
Axiom Math Says Verified Reasoning Can Outscale Informal AI
Carina Hong, founder and CEO of Axiom Math, argues on the AI for Science podcast that formal verification is not mainly a way to police AI errors but a mechanism for scaling reasoning itself. Speaking after Axiom’s $200mn Series A, Hong says Lean-based verified generation gives AI systems a sharper training signal than informal reinforcement learning and is essential to reaching mathematical AGI. She points to Axiom’s reported perfect score on the 2024 Putnam exam as evidence, while acknowledging that specification, provenance and human judgment remain hard limits.