Qwen3.8-Flash-Next Needs 172.78 GiB to Run 6 Billion Parameters

Qwen3.8-Flash-Next activates 6 billion parameters per token. That figure led every write-up of the launch, and it produced the same question across r/LocalLLM within hours: how much VRAM does it need? The answer is 172.78 GiB at FP8, or 335.28 GiB at BF16. Active parameter count does not reduce it.

Divide one by the other and you get the number worth keeping: 28.8 GiB of VRAM for every billion parameters the model computes with. A dense model sits near 1.0 on that measure. Sparsity buys speed and charges memory.

Why 6B active does not mean a 6B footprint

Flash-Next holds 512 experts. Each token routes to 10 of them plus one shared expert. Which 10 depends on the token, and the router can pick any of the 512 at any position. Every expert therefore sits in memory, ready, for a token that might never arrive.

The activation count describes arithmetic per token, which sets speed and price per token. Checkpoint size describes residency, which sets the hardware you need. Confusing the two is the most common mistake in reading a mixture-of-experts release, and framing the model around cost efficiency invites it.

A second layer compounds this. Alongside the 125B backbone, Flash-Next ships a 51B n-gram embedding table of 20 million bigram and trigram entries at layer 2, plus a 4B multi-token prediction module. Hugging Face lists the artifact as 180B parameters. The n-gram table can offload to host memory, which is the one place the design hands memory back.

How it compares to other sparse models

Flash-Next is the sparsest model in the group Qwen chose to compare itself against. The achievement and the cost sit in the same row.

Configurations that work

The vLLM recipe published with the model lists validated deployments. Two of its notes will cost you an afternoon if you miss them.

The 8x H200 line is the trap. The FP8 checkpoint uses 128-wide quantization blocks, and plain tensor parallelism across 8 GPUs does not divide them. Use TEP8 with expert parallelism enabled. Pipeline parallelism fails too, because the n-gram embedding layer has no pipeline-parallel implementation in the initial release.

Offloading the n-gram table with VLLM_PLE_CPU_OFFLOAD=1 needs at least 51 GB of host RAM plus headroom, and the offload runs on NVIDIA devices only at present.

What the memory buys

Qwen reports its sparse attention reaching up to 10.2x prefill and 6.6x decode attention-kernel speedups at one million tokens, and 8.6x the prefill throughput of Qwen3.7-Plus at a 90% prefix-cache hit rate. The native context runs to 262,144 tokens, and YaRN extends it to a million. Those are vendor figures on vendor hardware, and they are the return on the memory bill.

For most businesses the arithmetic ends before it starts. Qwen Cloud serves the production version at $0.16 per million input tokens and $0.47 per million output tokens. Self-hosting has to beat that after hardware, power, and the engineer who keeps TEP8 running. We looked at the wider version of this question in why open weights is not a price.

One licence clause before you deploy it for a client

The licence is Qwen Community License 1.0, not Apache 2.0, whatever some launch coverage said. Internal use inside your own product carries no restriction. Operating a model-as-a-service or AI work assistant business on these weights requires a separate licence from Qwen. Agencies standing this up for a client should read that clause before quoting the work.

When we scope AI automation, hosted inference wins on cost for almost every client below serious volume. The models worth self-hosting are the ones small enough to sit on hardware you already own, and 172.78 GiB is not that.