GPUs / By VRAM Best local LLMs for 32 GB VRAM (November 2026): what fits, context and speed With 32 GB of VRAM or unified memory, 71 models run well and 2 more are a tight fit. 34 don't fit.
Speeds assume 400 GB/s , the median memory bandwidth of the 16 GPUs and memory
configurations with exactly 32 GB. Yours may be faster or slower: pick your GPU for exact numbers.
Best overall Highest MMLU-Pro score among models that run well
Qwen3.6 27B Q8_0 62K context · ~13 tok/s · MMLU 86.2 Qwen3.5 27B Q8_0 88K context · ~14 tok/s · MMLU 86.1 Qwen3.5 35B A3B Q4_K_M 256K context · ~103 tok/s · MMLU 85.3 Gemma 4 31B Q8_0 21K context · ~12 tok/s · MMLU 85.2 Qwen3.6 35B A3B Q4_K_M 256K context · ~103 tok/s · MMLU 85.2 Best for coding Highest SWE-bench Verified score among models that run well
Qwen3.8 27B Q8_0 81K context · ~13 tok/s · SWE-bench 86 Ornith 1.5 35B A3B Q4_K_M 256K context · ~105 tok/s · SWE-bench 79 Qwen3.6 27B Q8_0 62K context · ~13 tok/s · SWE-bench 77.2 Muse Glimmer 30B Q8_0 128K context · ~13 tok/s · SWE-bench 76 Flint 40B A2.5B Q4_K_M 256K context · ~111 tok/s · SWE-bench 74.2 GPUs with 32 GB of memory Unified memory (Apple Silicon, Ryzen AI Max, RTX Spark) Every model in 32 GB 135 of 214 model configurations can run on 32 GB — sorted by quality
107 models, each counted once by the best way it runs on your hardware.
Largest that runs well Flint 40B A2.5B 39.79B params · Q4_K_M · 256K context Fastest good-quality model Phi-3 Mini 3.8B ~118 tok/s · Q4_K_M · 81K context Runs well 71 View all 71 →
Runs well 71 ✕ Q4_K_M Context 256K Speed ~23 tok/s
Q8_0 Context 62K Speed ~13 tok/s
Q4_K_M Context 256K Speed ~23 tok/s
Q8_0 Context 88K Speed ~14 tok/s
Q4_K_M Context 256K Speed ~103 tok/s
MoE — speed based on ~1.76 GB active experts plus per-token routing overhead (full 20.5 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 34.4 GB
Q4_K_M Context 256K Speed ~103 tok/s
MoE — speed based on ~1.77 GB active experts plus per-token routing overhead (full 20.61 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 34.4 GB
Q4_K_M Context 196K Speed ~21 tok/s
Q8_0 Context 21K Speed ~12 tok/s
Q4_K_M Context 55K Speed ~19 tok/s
Q8_0 Doesn't fit · needs 32.7 GB
Q4_K_M Context 16K Speed ~40 tok/s
Q8_0 Context 16K Speed ~24 tok/s
Q4_K_M Context 256K Speed ~23 tok/s
Q8_0 Context 81K Speed ~13 tok/s
Q4_K_M Context 55K Speed ~19 tok/s
Q8_0 Doesn't fit · needs 32.7 GB
QwQ 32B 32.5B Excellent Reasoning Tool use MMLU 83
Q4_K_M Context 55K Speed ~19 tok/s
Q8_0 Doesn't fit · needs 32.7 GB
Q4_K_M Context 256K Speed ~87 tok/s
MoE — speed based on ~2.44 GB active experts plus per-token routing overhead (full 15.87 GB loaded in VRAM)
Q8_0 Context 256K Speed ~65 tok/s
MoE — speed based on ~3.85 GB active experts plus per-token routing overhead (full 25.02 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~60 tok/s
Q8_0 Context 256K Speed ~38 tok/s
Q4_K_M Context 128K Speed ~106 tok/s
MoE — speed based on ~1.65 GB active experts plus per-token routing overhead (full 17.05 GB loaded in VRAM)
Q8_0 Context 46K Speed ~79 tok/s
MoE — speed based on ~2.87 GB active experts plus per-token routing overhead (full 29.66 GB loaded in VRAM)
Q4_K_M Context 121K Speed ~26 tok/s
Q8_0 Context 56K Speed ~15 tok/s
Q4_K_M Context 256K Speed ~91 tok/s
MoE — speed based on ~2.25 GB active experts plus per-token routing overhead (full 23.73 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 32.6 GB
Q4_K_M Context 160K Speed ~105 tok/s
MoE — speed based on ~1.7 GB active experts plus per-token routing overhead (full 17.28 GB loaded in VRAM)
Q8_0 Context 19K Speed ~77 tok/s
MoE — speed based on ~2.98 GB active experts plus per-token routing overhead (full 30.25 GB loaded in VRAM)
Q4_K_M Context 121K Speed ~26 tok/s
Q8_0 Context 56K Speed ~15 tok/s
Q4_K_M Context 55K Speed ~19 tok/s
Q8_0 Doesn't fit · needs 32.7 GB
Q4_K_M Context 128K Speed ~40 tok/s
Q8_0 Context 94K Speed ~24 tok/s
Q4_K_M Context 256K Speed ~89 tok/s
MoE — speed based on ~2.3 GB active experts plus per-token routing overhead (full 23.03 GB loaded in VRAM)
Q8_0 Context 119K Speed ~75 tok/s
MoE — speed based on ~3.13 GB active experts plus per-token routing overhead (full 31.28 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~107 tok/s
Q8_0 Context 256K Speed ~73 tok/s
Q4_K_M Context 128K Speed ~66 tok/s
Q8_0 Context 128K Speed ~41 tok/s
Q4_K_M Context 55K Speed ~19 tok/s
Q8_0 Doesn't fit · needs 32.7 GB
Q4_K_M Context 128K Speed ~23 tok/s
Q8_0 Context 69K Speed ~13 tok/s
Q4_K_M Context 256K Speed ~49 tok/s
Q8_0 Context 256K Speed ~29 tok/s
Q4_K_M Context 128K Speed ~47 tok/s
Q8_0 Context 128K Speed ~28 tok/s
Q4_K_M Context 128K Speed ~100 tok/s
MoE — speed based on ~1.86 GB active experts plus per-token routing overhead (full 10.83 GB loaded in VRAM)
Q8_0 Context 128K Speed ~98 tok/s
MoE — speed based on ~1.93 GB active experts plus per-token routing overhead (full 11.28 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~70 tok/s
Q8_0 Context 128K Speed ~44 tok/s
Q4_K_M Context 128K Speed ~70 tok/s
Q8_0 Context 128K Speed ~44 tok/s
Q4_K_M Context 128K Speed ~48 tok/s
Q8_0 Context 128K Speed ~30 tok/s
Q4_K_M Context 128K Speed ~40 tok/s
Q8_0 Context 94K Speed ~24 tok/s
Q4_K_M Context 128K Speed ~54 tok/s
Q8_0 Context 128K Speed ~33 tok/s
Q4_K_M Context 128K Speed ~40 tok/s
Q8_0 Context 94K Speed ~24 tok/s
Q4_K_M Context 121K Speed ~26 tok/s
Q8_0 Context 56K Speed ~15 tok/s
Q4_K_M Context 81K Speed ~118 tok/s
Q8_0 Context 77K Speed ~79 tok/s
Q4_K_M Context 128K Speed ~115 tok/s
Q8_0 Context 128K Speed ~79 tok/s
Q4_K_M Context 128K Speed ~68 tok/s
Q8_0 Context 128K Speed ~42 tok/s
fp16 Context 128K Speed ~23 tok/s
Q4_K_M Context 128K Speed ~62 tok/s
Q8_0 Context 128K Speed ~44 tok/s
Q4_K_M Context 128K Speed ~47 tok/s
Q8_0 Context 104K Speed ~29 tok/s
Q4_K_M Context 128K Speed ~70 tok/s
Q8_0 Context 128K Speed ~44 tok/s
Q4_K_M Context 128K Speed ~136 tok/s
Q8_0 Context 128K Speed ~93 tok/s
Q4_K_M Context 32K Speed ~136 tok/s
Q8_0 Context 32K Speed ~93 tok/s
Q4_K_M Context 128K Speed ~62 tok/s
Q8_0 Context 128K Speed ~42 tok/s
Q4_K_M Context 32K Speed ~129 tok/s
Q8_0 Context 32K Speed ~88 tok/s
Q4_K_M Context 128K Speed ~133 tok/s
Q8_0 Context 128K Speed ~90 tok/s
fp16 Context 128K Speed ~54 tok/s
Q4_K_M Context 128K Speed ~115 tok/s
Q8_0 Context 128K Speed ~78 tok/s
Q4_K_M Context 32K Speed ~28 tok/s
Q8_0 Context 32K Speed ~16 tok/s
Q4_K_M Context 128K Speed ~90 tok/s
Q8_0 Context 128K Speed ~67 tok/s
Q4_K_M Context 128K Speed ~70 tok/s
Q8_0 Context 128K Speed ~44 tok/s
Q4_K_M Context 256K Speed ~176 tok/s
Q8_0 Context 256K Speed ~133 tok/s
Q4_K_M Context 16K Speed ~37 tok/s
Q8_0 Context 16K Speed ~22 tok/s
Q4_K_M Context 16K Speed ~79 tok/s
Q8_0 Context 16K Speed ~49 tok/s
Q4_K_M Context 128K Speed ~222 tok/s
Q8_0 Context 128K Speed ~172 tok/s
Q4_K_M Context 256K Speed ~260 tok/s
Q8_0 Context 256K Speed ~220 tok/s
Q4_K_M Context 256K Speed ~101 tok/s
Q8_0 Context 256K Speed ~76 tok/s
Q4_K_M Context 256K Speed ~59 tok/s
Q8_0 Context 256K Speed ~37 tok/s
Q4_K_M Context 256K Speed ~105 tok/s
MoE — speed based on ~1.69 GB active experts plus per-token routing overhead (full 20.22 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 35.2 GB
Q4_K_M Context 256K Speed ~104 tok/s
MoE — speed based on ~1.72 GB active experts plus per-token routing overhead (full 19.14 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 33.2 GB
Q4_K_M Context 256K Speed ~111 tok/s
MoE — speed based on ~1.48 GB active experts plus per-token routing overhead (full 23.61 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 39.5 GB
Q4_K_M Context 256K Speed ~105 tok/s
MoE — speed based on ~1.68 GB active experts plus per-token routing overhead (full 19.71 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 34.4 GB
Q4_K_M Context 256K Speed ~58 tok/s
Q8_0 Context 256K Speed ~38 tok/s
Q4_K_M Context 256K Speed ~191 tok/s
Q8_0 Context 256K Speed ~141 tok/s
Arete 27B 27.36B Basic Vision Tool use Reasoning MMLU
Q4_K_M Context 256K Speed ~22 tok/s
Q8_0 Context 86K Speed ~13 tok/s
Q4_K_M Context 195K Speed ~61 tok/s
Q8_0 Context 168K Speed ~38 tok/s
Q4_K_M Context 128K Speed ~23 tok/s
Q8_0 Context 128K Speed ~13 tok/s
Q4_K_M Context 128K Speed ~150 tok/s
Q8_0 Context 128K Speed ~103 tok/s
Q4_K_M Context 128K Speed ~115 tok/s
MoE — speed based on ~1.38 GB active experts plus per-token routing overhead (full 13.53 GB loaded in VRAM)
Q8_0 Context 128K Speed ~87 tok/s
MoE — speed based on ~2.43 GB active experts plus per-token routing overhead (full 23.74 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~109 tok/s
MoE — speed based on ~1.55 GB active experts plus per-token routing overhead (full 7.52 GB loaded in VRAM)
Q8_0 Context 128K Speed ~86 tok/s
MoE — speed based on ~2.48 GB active experts plus per-token routing overhead (full 12.04 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~191 tok/s
Q8_0 Context 128K Speed ~141 tok/s
Q4_K_M Context 64K Speed ~56 tok/s
Q8_0 Context 64K Speed ~44 tok/s
Q4_K_M Context 63K Speed ~18 tok/s
Q8_0 Doesn't fit · needs 33.0 GB
Tight fit 2 Q4_K_M Context 8K Speed ~23 tok/s
Only 8K context — under the 16K needed to be practical
Q8_0 Context 8K Speed ~13 tok/s
Only 8K context — under the 16K needed to be practical
Q4_K_M Context 8K Speed ~59 tok/s
Only 8K context — under the 16K needed to be practical
Q8_0 Context 8K Speed ~37 tok/s
Only 8K context — under the 16K needed to be practical
Notify me when new models fit my GPU An email when a new model fits your 32 GB setup — no spam, unsubscribe anytime.
Doesn't fit 34 Q4_K_M Needs 381.2 GB — 349.2 GB over
Q8_0 Needs 664.4 GB — 632.4 GB over
Q4_K_M Needs 45.1 GB — 13.1 GB over
Q8_0 Needs 79.0 GB — 47.0 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 586.5 GB — 554.5 GB over
Q8_0 Needs 1013.0 GB — 981.0 GB over
View all 34 →
Doesn't fit 34 ✕ Q4_K_M Needs 381.2 GB — 349.2 GB over
Q8_0 Needs 664.4 GB — 632.4 GB over
Q4_K_M Needs 45.1 GB — 13.1 GB over
Q8_0 Needs 79.0 GB — 47.0 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 586.5 GB — 554.5 GB over
Q8_0 Needs 1013.0 GB — 981.0 GB over
Q4_K_M Needs 68.2 GB — 36.2 GB over
Q8_0 Needs 118.3 GB — 86.3 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 128.9 GB — 96.9 GB over
Q8_0 Needs 226.5 GB — 194.5 GB over
Q4_K_M Needs 900.1 GB — 868.1 GB over
Q8_0 Needs 1670.1 GB — 1638.1 GB over
Q4_K_M Needs 227.4 GB — 195.4 GB over
Q8_0 Needs 392.6 GB — 360.6 GB over
GLM-4.6 357B Excellent Reasoning Tool use MMLU 87
Q4_K_M Needs 200.8 GB — 168.8 GB over
Q8_0 Needs 353.3 GB — 321.3 GB over
Q4_K_M Needs 428.2 GB — 396.2 GB over
Q8_0 Needs 746.4 GB — 714.4 GB over
GLM 5.3 753.33B Excellent Tool use Reasoning MMLU 86.8
Q8_0 Needs 746.4 GB — 714.4 GB over
Q4_K_M Needs 161.1 GB — 129.1 GB over
Q8_0 Needs 282.1 GB — 250.1 GB over
Q4_K_M Needs 176.0 GB — 144.0 GB over
Q8_0 Needs 310.4 GB — 278.4 GB over
Q4_K_M Needs 44.5 GB — 12.5 GB over
Q8_0 Needs 72.3 GB — 40.3 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 44.5 GB — 12.5 GB over
Q8_0 Needs 72.3 GB — 40.3 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 39.9 GB — 7.9 GB over
Q8_0 Needs 70.1 GB — 38.1 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 226.3 GB — 194.3 GB over
Q8_0 Needs 396.8 GB — 364.8 GB over
Q4_K_M Needs 71.3 GB — 39.3 GB over
Q8_0 Needs 121.0 GB — 89.0 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 111.9 GB — 79.9 GB over
Q8_0 Needs 195.2 GB — 163.2 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 39.9 GB — 7.9 GB over
Q8_0 Needs 70.1 GB — 38.1 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
GLM 5 753.86B Excellent Tool use MMLU 83
Q4_K_M Needs 424.7 GB — 392.7 GB over
Q8_0 Needs 746.4 GB — 714.4 GB over
Q4_K_M Needs 175.9 GB — 143.9 GB over
Q8_0 Needs 306.8 GB — 274.8 GB over
Q4_K_M Needs 129.1 GB — 97.1 GB over
Q8_0 Needs 226.7 GB — 194.7 GB over
Q4_K_M Needs 61.1 GB — 29.1 GB over
Q8_0 Needs 106.9 GB — 74.9 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 111.4 GB — 79.4 GB over
Q8_0 Needs 175.4 GB — 143.4 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 379.1 GB — 347.1 GB over
Q8_0 Needs 666.6 GB — 634.6 GB over
Q4_K_M Needs 591.3 GB — 559.3 GB over
Q8_0 Needs 1038.6 GB — 1006.6 GB over
Q4_K_M Needs 465.3 GB — 433.3 GB over
Q8_0 Needs 811.9 GB — 779.9 GB over
Q4_K_M Needs 333.4 GB — 301.4 GB over
Q8_0 Needs 566.6 GB — 534.6 GB over
Q4_K_M Needs 293.0 GB — 261.0 GB over
Q8_0 Needs 496.2 GB — 464.2 GB over
Q4_K_M Needs 227.6 GB — 195.6 GB over
Q8_0 Needs 399.1 GB — 367.1 GB over
Q4_K_S Needs 104.0 GB — 72.0 GB over
Q8_0 Needs 195.2 GB — 163.2 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 70.7 GB — 38.7 GB over
Q8_0 Needs 123.5 GB — 91.5 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 65.3 GB — 33.3 GB over
Q8_0 Needs 119.7 GB — 87.7 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q8_0 Needs 73.4 GB — 41.4 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.