What GPU do you need to run GLM-4.6?
357B params 200K max context ReasoningTool use
The smallest GPU memory that runs GLM-4.6 well is 256 GB (Q4_K_M, e.g. M3 Ultra). 2 of 132 GPUs run it well.
MMLU 87 SWE-bench Verified 68.2
VRAM needed per quantization
| Quantization | Weights | 16K context | 32K context | Full 200K |
|---|---|---|---|---|
| Q4_K_M | 216.0 GB | 217.1 GB | 218.1 GB | 229.4 GB |
| Q8_0 | 379.0 GB | 380.1 GB | 381.1 GB | 392.4 GB |
Weights plus fp16 KV cache. Leave ~1 GB headroom for the inference engine.
GPUs that can run GLM-4.6
Best quantization that fits on each GPU, with the context and speed it gets. For chips with several memory sizes, the smallest that works is shown.