Hugging Face Publishes 207 Device-Adaptive WebGPU Kernels

Nico Martin presents @huggingface/kernels as a way to invoke versioned GPU operations from JavaScript without choosing or writing the shader implementation yourself. A developer names a kernel on the Hugging Face Hub, requests a contract version, supplies typed-array inputs, and awaits the result.

The abstraction rests on a distinction between an operation and a kernel. An operation is the portable mathematical contract. For addition, it specifies $C=A+B$: expected inputs and outputs, shapes, types, and semantics such as broadcasting. It does not prescribe a shader language, hardware target, or workgroup size.

A kernel is the executable implementation of that operation for a particular platform. In a browser, that means WGSLWebGPU Shading Language, the shader language browsers validate and compile for a GPU.. The same operation could instead have a CUDA or Metal kernel on other platforms. That separation lets an application depend on a versioned, portable operation contract while execution details vary with the machine.

A model is, in Martin’s description, a graph of operations: additions, matrix multiplications, convolutions, normalizations, and others. The operation fixes the math. The kernel determines how that math is executed on a particular device.

The project does not publish one frozen WGSL file as the definitive implementation of an operation. It publishes Jinja templates. The repository for its add operation, for example, includes templates for ordinary binary addition, broadcasting, and vectorized execution. Those templates can be rendered with different element types, vector widths, and workgroup sizes.

Martin’s constraint is straightforward: “optimal” varies by WebGPU adapter. A broadly compatible implementation may leave performance on the table on a device that supports wider data layouts or larger workgroups. The library therefore chooses a variant the current device supports rather than assuming a single shader is best everywhere.

Two rendered versions of the same add operation illustrate the trade-off. One uses scalar f32 lanes and a workgroup size of 64. Another uses vec4<f32> values and a workgroup size of 256, so one invocation can handle four values. Both implement addition; Martin says the latter runs more efficiently when the device supports the wider type.

That device-specific work is what the JavaScript API hides. The loader fetches the versioned manifest and templates, validates the operation contract and inputs, inspects WebGPU capabilities, selects a valid variant, renders and compiles WGSL, manages buffers, dispatches the work, and returns an asynchronous executable result.

The package is presented as an early preview: its API may change and kernel coverage is still growing. A requested version can follow the repository’s moving version branch, while a 40-character commit revision can pin kernel bytes that will not change.

Nico Martin uses the attention demonstration to show that the library’s value is not limited to calling one add function. It composes matrix multiplication, softmax, add, and layer normalization into a transformer attention block in roughly 20 lines of JavaScript.

Attention gives a token representation context from surrounding tokens. Text becomes token IDs and vector representations, but the model must also calculate how strongly tokens relate to one another. In the heatmap for the sentence “The small robot carried the red book across the quiet library,” each row represents a token looking at others; brighter cells indicate greater attention. “Carried” shows a strong connection to “robot.”

The demo uses pretrained tinyBERT weights and builds the heatmap locally with WebGPU rather than showing a precomputed diagram. Its code requests GPU output for intermediate tensors, allowing one kernel’s result to flow directly into the next kernel without returning it to JavaScript for CPU readback. That is the operational advantage of composition: a chain of tensor operations can remain in GPU memory instead of repeatedly crossing the JavaScript–GPU boundary.

The same kernel layer also drives a non-model example: a 1,024-by-1,024 animated field with more than one million cells. It combines 128 moving wave channels through one matrix multiplication. One matrix holds row-specific wave amplitudes that change every frame; a precomputed second matrix holds horizontal wave patterns. Their product produces the complete field in one kernel call.

The demo describes each output pixel as a dot product over 128 wave values, for about 134 million calculations. Its interface reports 60 display frames per second and an uncapped estimate of roughly 426 kernel runs per second. Martin attributes the 60fps display rate to the browser’s requestAnimationFrame cap; he says the same calculation in plain JavaScript drops to roughly 5–7 frames per second.

A smaller interactive sphere applies the same idea: one matrix multiplication calculates color values for roughly 8,000 surface cells.

Nico Martin frames device-specific rendering as a testing constraint. Hugging Face can optimize kernels on hardware its team has access to, he says, but there are thousands of devices in use. The performance of a kernel can differ across adapters, and broader measurements can reveal cases the team did not anticipate.

Fleet asks users to benchmark their own GPU in the browser. It loads kernels from the Hub, runs WebGPU benchmarks, and returns a device rating card to the participant. For Hugging Face, the results provide performance data across hardware it cannot directly test, which can inform further optimization of templates and variant choices.

The Apple M3 Max card shown reports a score of 92, a “top 2%” label among ranked WebGPU devices, 103 operations estimated per second, 88% of operations estimated, and a 1:03 total trial time.

Martin describes an open-source feedback loop rather than a universal best shader: measurements from more devices can help identify where performance needs work, and subsequent optimizations can improve the kernels made available across that broader range of hardware.