What Models?

Pick the right model for your GPU — in seconds.

Llama 4 Maverick VRAM requirements: which GPUs can run it?

400B params 128K max context VisionTool use

The smallest GPU memory that runs Llama 4 Maverick well is 256 GB (Q4_K_M, e.g. M3 Ultra). 2 of 132 GPUs run it well. It's a mixture-of-experts model, so all weights must fit in memory but speed depends only on the active experts.

MMLU 85.5

VRAM needed per quantization

QuantizationWeights16K context32K contextFull 128K
Q4_K_M226.1 GB229.0 GB231.9 GB249.5 GB
Q8_0396.6 GB399.5 GB402.4 GB420.0 GB

Weights plus fp16 KV cache. Leave ~1 GB headroom for the inference engine.

Run it locally

Each command downloads the GGUF from Hugging Face, then Ollama starts a chat and llama.cpp starts an OpenAI-compatible server with a web UI on port 8080. Both fetch the vision projector (mmproj) automatically.

llama.cpp llama-server -hf unsloth/Llama-4-Maverick-17B-128E-Instruct-GGUF:Q4_K_M

Split into 5 files, which Ollama can't pull from Hugging Face yet; llama.cpp downloads the whole set.

llama.cpp llama-server -hf unsloth/Llama-4-Maverick-17B-128E-Instruct-GGUF:Q8_0

Split into 9 files, which Ollama can't pull from Hugging Face yet; llama.cpp downloads the whole set.

GPUs that can run Llama 4 Maverick

Best quantization that fits on each GPU, with the context and speed it gets. For chips with several memory sizes, the smallest that works is shown.

Apple

  • M3 Ultra 256 GB Q4_K_M 128K context · ~45 tok/s
  • M5 Ultra 256 GB Q4_K_M 128K context · ~63 tok/s
Check your exact setup in the full tool →