What GPU do you need to run Mistral Large 3 675B?
673.42B params 288K max context VisionTool useReasoning
The smallest GPU memory that runs Mistral Large 3 675B well is 512 GB (Q4_K_M, e.g. M3 Ultra). 2 of 132 GPUs run it well. It's a mixture-of-experts model, so all weights must fit in memory but speed depends only on the active experts.
MMLU 73.11
VRAM needed per quantization
| Quantization | Weights | 16K context | 32K context | Full 288K |
|---|---|---|---|---|
| Q4_K_M | 379.0 GB | 380.1 GB | 381.1 GB | 397.8 GB |
| Q8_0 | 666.5 GB | 667.6 GB | 668.6 GB | 685.3 GB |
Weights plus fp16 KV cache. Leave ~1 GB headroom for the inference engine.
GPUs that can run Mistral Large 3 675B
Best quantization that fits on each GPU, with the context and speed it gets. For chips with several memory sizes, the smallest that works is shown.