Qwen Flash-Next Matches Larger Models With 6B Active Parameters
Károly Zsolnai-Fehér argues that Qwen3.8-Flash-Next, a 125-billion-parameter open-weights mixture-of-experts model, is approaching the performance of far larger paid and proprietary systems while activating only 6 billion parameters per token. He attributes that position to changes aimed at reducing long-context costs and improving information flow, and points to early benchmarks and local runs as evidence that competitive models can increasingly be downloaded and operated outside subscription APIs.
A 125B-parameter model is being positioned against much larger systems
Károly Zsolnai-Fehér Károly Zsolnai-Fehér argues that Qwen3.8-Flash-Next is an unusually consequential open-weights release not simply because it is free to download, but because its early results put it in reach of substantially larger models. His framing is practical: paid, closed systems can increasingly be substituted with models people can download and run themselves, without a subscription.
Flash-Next sits between Qwen’s 27-billion-parameter dense model and Qwen3.8 Max. The distinction is not merely scale. The source identifies the 27B model as dense, while Flash-Next is a mixture-of-experts model: it has 125 billion parameters in total, but activates 6 billion for each token. Qwen3.8 Max, by comparison, is displayed as having 2.4 trillion total parameters, 95 billion active parameters, and a 51-billion-parameter N-gram embedding.
That selective activation, Zsolnai-Fehér says, suits systems with ample but relatively slow memory bandwidth, such as Nvidia DGX Sparks. He reports running Flash-Next on two Sparks at roughly 38 tokens per second on the first day. A run displayed from Alok_spot uses an RTX 4090 with 24 GB of VRAM and reports 22 tokens per second, plus 350 tokens per second during prefill in llama.cpp.
The demonstrations are qualitative as well as throughput-oriented. A comparison credited to Loktar places a Qwen-produced Warcraft III menu beside the 2002 original; another local run, credited to Ok_Ninja7526, shows a dark game-like corridor. In a web-animation prompt for a realistic solar-system visualization, a comparison credited to Alok shows Flash-Next producing an output in 8,414 tokens against 9,642 for Qwen 3.8 27B, with reasoning turned off in both cases.
| Model | Total parameters | Active parameters | N-gram embedding | Reported positioning |
|---|---|---|---|---|
| Qwen3.8 27B | 27B | 27B | — | Dense model |
| Qwen3.8-Flash-Next | 125B | 6B | — | Mixture of experts |
| Qwen3.8 Max | 2.4T | 95B | 51B | Largest model shown |
The design changes target long-context cost and information flow
As documents accumulate or a conversation continues, the model’s working context fills up. Zsolnai-Fehér describes conventional full attention as having quadratic complexity: doubling the context means roughly quadrupling the work.
The first of the model’s three highlighted changes is Qwen Sparse Attention, or QSA. He contrasts it with DeepSeek’s DSA, which selects important individual tokens to reduce attention work. QSA instead groups tokens into small blocks and searches only the relevant blocks. Zsolnai-Fehér’s stated reason is that grouping makes growing context cheaper.
The second change, Gated Residual, concerns the additional information the model carries for each token. In his account, layers can interfere with one another when they continually rewrite the same running information. Gated Residual uses four branches rather than one, so that some information can be left alone while other information is changed.
The third is N-gram embedding. Zsolnai-Fehér uses “hot dog” to illustrate why short multi-token combinations warrant their own representation: “hot” and “dog” separately do not carry the same meaning as the phrase. Flash-Next groups such combinations and builds lookup memory intended for cheap, quick retrieval.
He characterizes this as a variant of a technique DeepSeek already uses, with an implementation difference: DeepSeek distributes lookup memory across multiple layers, while Qwen places one large lookup layer near the beginning of the network.
The displayed ranking makes efficiency the point
An Artificial Analysis Intelligence Index chart shown in the source gives Qwen3.8 Flash-Next a score of 56. It places the model below Claude Fable 5 with fallback, at 62, and above DeepSeek V4 Pro 0813 at 53, Qwen 27B at 52, and GPT-5.6 Luna at 52.
The chart makes the size comparison conspicuous. Flash-Next is marked “FREE,” with 125B total parameters and 6B active parameters, and its bar is labelled “~13x smaller.” DeepSeek V4 Pro is shown with 1.6T total parameters and 498B active parameters. Claude Fable 5 and GPT-5.6 Luna are marked as paid systems.
Zsolnai-Fehér says the three architectural changes together yield a system that already outperforms some leading open-weights models and may even exceed the much larger DeepSeek V4 Pro. He also emphasizes the timing: these are results seen only a few days after release.
The comparison does not establish that hardware constraints have disappeared. It instead makes a more operational proposition: a model need not activate anything close to its full parameter count on every token to be competitive with much larger systems.