Mistral Large 4 1T A52B VRAM requirements: which GPUs can run it?
Mistral Large 4 1T A52B needs at least 592.3 GB (Q4_K_M, 16K context), more than any single GPU in our database (the largest is M3 Ultra with 512 GB). Try multiple GPUs or system RAM offloading in the full tool. It's a mixture-of-experts model, so all weights must fit in memory but speed depends only on the active experts.
VRAM needed per quantization
| Quantization | Weights | 16K context | 32K context | Full 1M |
|---|---|---|---|---|
| Q4_K_M | 591.2 GB | 592.3 GB | 593.4 GB | 660.8 GB |
| Q8_0 | 1038.6 GB | 1039.6 GB | 1040.7 GB | 1108.2 GB |
Weights plus fp16 KV cache. Leave ~1 GB headroom for the inference engine.
Run it locally
Each command downloads the GGUF from Hugging Face, then Ollama starts a chat and llama.cpp starts an OpenAI-compatible server with a web UI on port 8080. Both fetch the vision projector (mmproj) automatically.
llama-server -hf ggml-org/Mistral-Large-4-1T-A52B-GGUF:Q4_K_M Split into 13 files, which Ollama can't pull from Hugging Face yet; llama.cpp downloads the whole set.
llama-server -hf ggml-org/Mistral-Large-4-1T-A52B-GGUF:Q8_0 Split into 24 files, which Ollama can't pull from Hugging Face yet; llama.cpp downloads the whole set.
GPUs that can run Mistral Large 4 1T A52B
None on its own: it needs more memory than the largest single GPU here (512 GB). Open the tool to try multiple GPUs or system RAM offloading.