VRAM by precision
Source: vLLM recipe, 8 September 2026. Bytes-per-parameter is multiplied by 2.4T total parameters (95B active per token). Numbers are the parameter-only weight footprint before KV cache, activations, or runtime overhead.
| Precision | Weights | Sized | B300 (268 GB) | MI355X (288 GB) | H200 (141 GB) | GB300 tray |
|---|---|---|---|---|---|---|
| BF16 | 4.45 TiB | 5,871 GB | 24 GPUs | 24 GPUs | 48 GPUs | 6 trays |
| FP8 | 2.27 TiB | 2,996 GB | 16 GPUs | 16 GPUs | 32 GPUs | 4 trays (TP16) |
NVIDIA Dynamo production recipe
NVIDIA Dynamo's reference serving topology for this model: 16× GB300 across 4 nodes, TP16 over MNNVL, FP8 weights and FP8 KV cache. Confirms the vLLM FP8 estimate and is the configuration to benchmark against if you are self-hosting.
How these figures are calculated
Bytes-per-parameter is applied to 2.4T total parameters (95B active per token). Numbers are the parameter-only weight footprint before KV cache, activations, or runtime overhead. Multiply by ~1.2 to budget for sustained inference at the 262K native context, and by ~1.4 if you intend to extend to the 1,010,000-token extended context.
Quantized variants and derivative releases
RadixArk's NVFP4 quantization is already published on Hugging Face — derivative release date 7 August 2026, ahead of the official weights. NVFP4 brings the FP8 footprint down further; check the derivative listing for the actual weight size and any quality caveats before deploying.
Ready to test the workflow?
Weights are publicly available
Qwen3.8-2.4T-A95B weights are released under the qwen3.8-max license and can be pulled from the Hugging Face repository linked in Sources. Confirm the license tag and any use restrictions against the official repository before self-hosting at scale.
Architecture
2.4T-parameter Mixture-of-Experts (95B active per token), native 262,144-token context, extensible to 1,010,000 tokens via YaRN. Text-only input on this checkpoint.