MiniCPM-V 4.7 35B A3B VRAM requirements: which GPUs can run it?
The smallest GPU memory that runs MiniCPM-V 4.7 35B A3B well is 24 GB (Q4_K_M, e.g. RTX Spark). 45 of 132 GPUs run it well and 2 more are a tight fit. It's a mixture-of-experts model, so all weights must fit in memory but speed depends only on the active experts.
VRAM needed per quantization
| Quantization | Weights | 16K context | 32K context | Full 256K |
|---|---|---|---|---|
| Q4_K_M | 19.7 GB | 20.0 GB | 20.3 GB | 24.6 GB |
| Q8_0 | 34.4 GB | 34.7 GB | 35.0 GB | 39.2 GB |
Weights plus fp16 KV cache. Leave ~1 GB headroom for the inference engine.
Run it locally
Each command downloads the GGUF from Hugging Face, then Ollama starts a chat and llama.cpp starts an OpenAI-compatible server with a web UI on port 8080. Both fetch the vision projector (mmproj) automatically.
ollama run hf.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf:Q4_K_M llama-server -hf openbmb/MiniCPM-V-4.7-35B-A3B-gguf:Q4_K_M ollama run hf.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf:Q8_0 llama-server -hf openbmb/MiniCPM-V-4.7-35B-A3B-gguf:Q8_0 Cheapest GPUs that can run MiniCPM-V 4.7 35B A3B
Lowest launch MSRP among GPUs that run its smallest quantization (Q4_K_M) well, with at least 16K context.
As an Amazon Associate, I earn from qualifying purchases.
GPUs that can run MiniCPM-V 4.7 35B A3B
Best quantization that fits on each GPU, with the context and speed it gets. For chips with several memory sizes, the smallest that works is shown.
NVIDIA
- RTX 4000 Ada 20 GB Q4_K_M 15K context · ~97 tok/s tight fit
- RTX 3090 24 GB Q4_K_M 225K context · ~182 tok/s
- RTX 3090 Ti 24 GB Q4_K_M 225K context · ~189 tok/s
- RTX 4090 24 GB Q4_K_M 225K context · ~189 tok/s
- RTX 4500 Ada 24 GB Q4_K_M 225K context · ~111 tok/s
- RTX A5000 24 GB Q4_K_M 225K context · ~163 tok/s
- RTX PRO 4000 Blackwell 24 GB Q4_K_M 225K context · ~150 tok/s
- RTX Spark 24 GB Q4_K_M 225K context · ~84 tok/s
- Tesla P40 24 GB Q4_K_M 225K context · ~94 tok/s
- RTX 5090 32 GB Q4_K_M 256K context · ~246 tok/s
- RTX 5000 Ada 32 GB Q4_K_M 256K context · ~136 tok/s
- RTX PRO 4500 Blackwell 32 GB Q4_K_M 256K context · ~178 tok/s
- Tesla V100 32GB 32 GB Q4_K_M 256K context · ~178 tok/s
- L40S 48 GB Q8_0 256K context · ~137 tok/s
- RTX 6000 Ada 48 GB Q8_0 256K context · ~147 tok/s
- RTX A6000 48 GB Q8_0 256K context · ~127 tok/s
- RTX PRO 5000 Blackwell 48 GB Q8_0 256K context · ~179 tok/s
- RTX PRO 5000 Blackwell 72GB 72 GB Q8_0 256K context · ~179 tok/s
- RTX PRO 5500 Blackwell 84 GB Q8_0 256K context · ~183 tok/s
- RTX PRO 6000 Blackwell 96 GB Q8_0 256K context · ~208 tok/s
- DGX Spark (GB 10) 128 GB Q8_0 256K context · ~57 tok/s
AMD
- Radeon RX 7900 XT 20 GB Q4_K_M 15K context · ~167 tok/s tight fit
- Radeon RX 7900 XTX 24 GB Q4_K_M 225K context · ~184 tok/s
- Radeon AI PRO R9700 32 GB Q4_K_M 256K context · ~145 tok/s
- Ryzen AI Max (Strix Halo) 32 GB Q4_K_M 256K context · ~74 tok/s
- Ryzen AI Max 400 (Gorgon Halo) 64 GB Q8_0 256K context · ~57 tok/s
Intel
- Arc Pro B65 32 GB Q4_K_M 256K context · ~141 tok/s
- Arc Pro B70 32 GB Q4_K_M 256K context · ~141 tok/s
Apple
- M2 24 GB Q4_K_M 225K context · ~33 tok/s
- M3 24 GB Q4_K_M 225K context · ~33 tok/s
- M4 24 GB Q4_K_M 225K context · ~39 tok/s
- M4 Pro 24 GB Q4_K_M 225K context · ~78 tok/s
- M5 24 GB Q4_K_M 225K context · ~48 tok/s
- M5 Pro 24 GB Q4_K_M 225K context · ~86 tok/s
- M6 24 GB Q4_K_M 225K context · ~53 tok/s
- M1 Max 32 GB Q4_K_M 256K context · ~105 tok/s
- M1 Pro 32 GB Q4_K_M 256K context · ~61 tok/s
- M2 Max 32 GB Q4_K_M 256K context · ~105 tok/s
- M2 Pro 32 GB Q4_K_M 256K context · ~61 tok/s
- M3 Max 36 GB Q8_0 85K context · ~61 tok/s
- M3 Pro 36 GB Q8_0 85K context · ~33 tok/s
- M4 Max 36 GB Q8_0 85K context · ~79 tok/s
- M5 Max 36 GB Q8_0 85K context · ~87 tok/s
- M1 Ultra 64 GB Q8_0 256K context · ~101 tok/s
- M2 Ultra 64 GB Q8_0 256K context · ~101 tok/s
- M3 Ultra 96 GB Q8_0 256K context · ~103 tok/s
- M5 Ultra 96 GB Q8_0 256K context · ~134 tok/s