Dan Fu

Assistant Professor of Computer Science and Engineering at UC San Diego and VP of Kernels at Together AI, focused on efficient machine learning systems, GPU kernels, hardware-aware algorithms, and scalable ML architectures.

Inference Constraints Are Reshaping Language Model Architecture

In a Stanford CS336 guest lecture, Dan Fu argued that language-model inference is no longer downstream plumbing but a central research and design constraint. Fu described serving as the machinery that turns a trained model into a usable system, where schedulers, KV caches, GPU kernels, routing policies and hardware choices determine which architectures are practical, economical and reliable at scale.

Stanford OnlineJun 5, 202622 min read