What GPU do you need to run GLM 5?
753.86B params 198K max context Tool use
The smallest GPU memory that runs GLM 5 well is 512 GB (Q4_K_M, e.g. M3 Ultra). 2 of 132 GPUs run it well.
MMLU 83 SWE-bench Verified 77.8
VRAM needed per quantization
| Quantization | Weights | 16K context | 32K context | Full 198K |
|---|---|---|---|---|
| Q4_K_M | 424.6 GB | 425.9 GB | 427.3 GB | 441.2 GB |
| Q8_0 | 746.3 GB | 747.7 GB | 749.0 GB | 762.9 GB |
Weights plus fp16 KV cache. Leave ~1 GB headroom for the inference engine.
GPUs that can run GLM 5
Best quantization that fits on each GPU, with the context and speed it gets. For chips with several memory sizes, the smallest that works is shown.