VRAM by precision
Source: vLLM recipe, 8 September 2026. Bytes-per-parameter is multiplied by 2.4T total parameters (95B active per token). Numbers are the parameter-only weight footprint before KV cache, activations, or runtime overhead.
| Precision | Weights | Sized | B300 (268 GB) | MI355X (288 GB) | H200 (141 GB) | GB300 tray |
|---|---|---|---|---|---|---|
| BF16 | 4.45 TiB | 5,871 GB | 24 GPUs | 24 GPUs | 48 GPUs | 6 trays |
| FP8 | 2.27 TiB | 2,996 GB | 16 GPUs | 16 GPUs | 32 GPUs | 4 trays (TP16) |
NVIDIA Dynamo production recipe
NVIDIA Dynamo's reference serving topology for this model: 16× GB300 across 4 nodes, TP16 over MNNVL, FP8 weights and FP8 KV cache. Confirms the vLLM FP8 estimate and is the configuration to benchmark against if you are self-hosting.
How these figures are calculated
Bytes-per-parameter is applied to 2.4T total parameters (95B active per token). Numbers are the parameter-only weight footprint before KV cache, activations, or runtime overhead. Multiply by ~1.2 to budget for sustained inference at the 262K native context, and by ~1.4 if you intend to extend to the 1,010,000-token extended context.
Quantized variants and derivative releases
RadixArk's NVFP4 quantization is already published on Hugging Face — derivative release date 7 August 2026, ahead of the official weights. NVFP4 brings the FP8 footprint down further; check the derivative listing for the actual weight size and any quality caveats before deploying.
Ready to test the workflow?
Create account & add creditsWeights are publicly available
Qwen3.8-2.4T-A95B weights are released under the qwen3.8-max license and can be pulled from the Hugging Face repository linked in Sources. Confirm the license tag and any use restrictions against the official repository before self-hosting at scale.
Architecture
2.4T-parameter Mixture-of-Experts (95B active per token), native 262,144-token context, extensible to 1,010,000 tokens via YaRN. Text-only input on this checkpoint.
Frequently asked questions
What GPU do I need to run Qwen3.8-2.4T-A95B?
At FP8: 16× B300 (268 GB each), 16× MI355X (288 GB each), or 32× H200 (141 GB each). At BF16: 24× B300, 24× MI355X, or 48× H200. NVIDIA Dynamo's production recipe runs 16× GB300 across 4 nodes at TP16 with FP8 weights and FP8 KV cache.
Can Qwen3.8-2.4T-A95B run on 16GB VRAM?
No. The FP8 weights alone are 2.96 TB. Even a heavily quantized INT4 build sits well above what consumer 16 GB cards can hold, and the 95B-active MoE routing makes any single-GPU serving impractical regardless.
How is VRAM estimated?
Bytes-per-parameter is multiplied by 2.4T total parameters (95B active per token). The figures are the parameter-only weight footprint before KV cache, activations, or runtime overhead — multiply by ~1.2 for sustained 262K-context inference and ~1.4 for the 1M extended context.
Where is the live Qwen3.8-2.4T-A95B model page?
The canonical model page with current OneInfer pricing, capabilities, and availability is /models/Qwen/Qwen3.8-2.4T-A95B. This page is a focused facet of that entity, not a replacement for it.
How should I treat benchmark or price claims?
Check each claim's provenance label and observed date. Vendor-reported and independently verified numbers are shown as separate evidence classes on this hub.
Is Qwen3.8-2.4T-A95B open source?
Yes — weights are released under the qwen3.8-max license. Confirm the license tag against the official Hugging Face repository before self-hosting at scale.
Put Qwen3.8-2.4T-A95B to work
Fund a controlled evaluation, start with a prepared prompt, and measure quality and cost on your own workload.