What GPU do you need to run Mistral Small 4?
119B params 1M max context VisionTool use
The smallest GPU memory that runs Mistral Small 4 well is 84 GB (Q4_K_M, e.g. RTX PRO 5500 Blackwell). 14 of 132 GPUs run it well.
MMLU 88.5
VRAM needed per quantization
| Quantization | Weights | 16K context | 32K context | Full 1M |
|---|---|---|---|---|
| Q4_K_M | 72.6 GB | 81.4 GB | 90.2 GB | 634.8 GB |
| Q8_0 | 126.5 GB | 135.3 GB | 144.0 GB | 688.6 GB |
Weights plus fp16 KV cache. Leave ~1 GB headroom for the inference engine.
Cheapest GPUs that can run Mistral Small 4
Lowest launch MSRP among GPUs that run its smallest quantization (Q4_K_M) well, with at least 16K context.
As an Amazon Associate, I earn from qualifying purchases.
GPUs that can run Mistral Small 4
Best quantization that fits on each GPU, with the context and speed it gets. For chips with several memory sizes, the smallest that works is shown.
NVIDIA
- RTX PRO 5500 Blackwell 84 GB Q4_K_M 20K context · ~19 tok/s
- RTX PRO 6000 Blackwell 96 GB Q4_K_M 42K context · ~24 tok/s
- DGX Spark (GB 10) 128 GB Q4_K_M 100K context · ~4 tok/s
- RTX Spark 128 GB Q4_K_M 100K context · ~4 tok/s
AMD
- Ryzen AI Max (Strix Halo) 96 GB Q4_K_M 42K context · ~3 tok/s
- Ryzen AI Max 400 (Gorgon Halo) 128 GB Q4_K_M 100K context · ~4 tok/s
Apple
- M2 Max 96 GB Q4_K_M 42K context · ~5 tok/s
- M3 Max 96 GB Q4_K_M 42K context · ~4 tok/s
- M3 Ultra 96 GB Q4_K_M 42K context · ~11 tok/s
- M5 Ultra 96 GB Q4_K_M 42K context · ~16 tok/s
- M1 Ultra 128 GB Q4_K_M 100K context · ~11 tok/s
- M2 Ultra 128 GB Q4_K_M 100K context · ~11 tok/s
- M4 Max 128 GB Q4_K_M 100K context · ~7 tok/s
- M5 Max 128 GB Q4_K_M 100K context · ~8 tok/s