GPUs / By VRAM Best local LLMs for 128 GB VRAM (November 2026): what fits, context and speed With 128 GB of VRAM or unified memory, 85 models run well and 2 more are a tight fit. 20 don't fit.
Speeds assume 400 GB/s , the median memory bandwidth of the 9 GPUs and memory
configurations with exactly 128 GB. Yours may be faster or slower: pick your GPU for exact numbers.
Best for coding Highest SWE-bench Verified score among models that run well
Qwen3.8 27B Q8_0 256K context · ~13 tok/s · SWE-bench 86 Ornith 1.5 122B Q8_0 197K context · ~32 tok/s · SWE-bench 84.5 Ornith 1.5 35B A3B Q8_0 256K context · ~78 tok/s · SWE-bench 79 Qwen3.6 27B Q8_0 256K context · ~13 tok/s · SWE-bench 77.2 Step 3.7 Flash Q4_K_S 140K context · ~49 tok/s · SWE-bench 76.5 GPUs with 128 GB of memory Unified memory (Apple Silicon, Ryzen AI Max, RTX Spark) Every model in 128 GB 172 of 214 model configurations can run on 128 GB — sorted by quality
107 models, each counted once by the best way it runs on your hardware.
Largest that runs well Step 3.7 Flash 198B params · Q4_K_S · 140K context Fastest good-quality model Phi-3 Mini 3.8B ~118 tok/s · Q4_K_M · 128K context Runs well 85 Q4_K_M Context 256K Speed ~105 tok/s
MoE — speed based on ~1.69 GB active experts plus per-token routing overhead (full 45.09 GB loaded in VRAM)
Q8_0 Context 256K Speed ~77 tok/s
MoE — speed based on ~2.96 GB active experts plus per-token routing overhead (full 78.99 GB loaded in VRAM)
Q4_K_M Context 109K Speed ~67 tok/s
MoE — speed based on ~3.7 GB active experts plus per-token routing overhead (full 67.65 GB loaded in VRAM)
Q8_0 Context 18K Speed ~45 tok/s
MoE — speed based on ~6.43 GB active experts plus per-token routing overhead (full 117.79 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~23 tok/s
Q8_0 Context 256K Speed ~13 tok/s
View all 85 →
Runs well 85 ✕ Q4_K_M Context 256K Speed ~105 tok/s
MoE — speed based on ~1.69 GB active experts plus per-token routing overhead (full 45.09 GB loaded in VRAM)
Q8_0 Context 256K Speed ~77 tok/s
MoE — speed based on ~2.96 GB active experts plus per-token routing overhead (full 78.99 GB loaded in VRAM)
Q4_K_M Context 109K Speed ~67 tok/s
MoE — speed based on ~3.7 GB active experts plus per-token routing overhead (full 67.65 GB loaded in VRAM)
Q8_0 Context 18K Speed ~45 tok/s
MoE — speed based on ~6.43 GB active experts plus per-token routing overhead (full 117.79 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~23 tok/s
Q8_0 Context 256K Speed ~13 tok/s
Q4_K_M Context 256K Speed ~23 tok/s
Q8_0 Context 256K Speed ~14 tok/s
Q4_K_M Context 128K Speed ~8 tok/s
Q8_0 Context 128K Speed ~5 tok/s
Q4_K_M Context 128K Speed ~8 tok/s
Q8_0 Context 128K Speed ~5 tok/s
Q4_K_M Context 128K Speed ~9 tok/s
Q8_0 Context 128K Speed ~5 tok/s
Q4_K_M Context 256K Speed ~103 tok/s
MoE — speed based on ~1.76 GB active experts plus per-token routing overhead (full 20.5 GB loaded in VRAM)
Q8_0 Context 256K Speed ~77 tok/s
MoE — speed based on ~2.95 GB active experts plus per-token routing overhead (full 34.37 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~21 tok/s
Q8_0 Context 256K Speed ~12 tok/s
Q4_K_M Context 256K Speed ~103 tok/s
MoE — speed based on ~1.77 GB active experts plus per-token routing overhead (full 20.61 GB loaded in VRAM)
Q8_0 Context 256K Speed ~77 tok/s
MoE — speed based on ~2.95 GB active experts plus per-token routing overhead (full 34.37 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~48 tok/s
MoE — speed based on ~5.84 GB active experts plus per-token routing overhead (full 71.28 GB loaded in VRAM)
Q8_0 Context 256K Speed ~32 tok/s
MoE — speed based on ~9.91 GB active experts plus per-token routing overhead (full 120.95 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~19 tok/s
Q8_0 Context 128K Speed ~11 tok/s
Q4_K_M Context 16K Speed ~40 tok/s
Q8_0 Context 16K Speed ~24 tok/s
Q4_K_M Context 256K Speed ~23 tok/s
Q8_0 Context 256K Speed ~13 tok/s
Q4_K_M Context 128K Speed ~9 tok/s
Q8_0 Context 128K Speed ~5 tok/s
Q4_K_M Context 94K Speed ~46 tok/s
MoE — speed based on ~6.24 GB active experts plus per-token routing overhead (full 111.7 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 195.2 GB
Q4_K_M Context 128K Speed ~19 tok/s
Q8_0 Context 128K Speed ~11 tok/s
QwQ 32B 32.5B Excellent Reasoning Tool use MMLU 83
Q4_K_M Context 128K Speed ~19 tok/s
Q8_0 Context 128K Speed ~11 tok/s
Q4_K_M Context 256K Speed ~87 tok/s
MoE — speed based on ~2.44 GB active experts plus per-token routing overhead (full 15.87 GB loaded in VRAM)
Q8_0 Context 256K Speed ~65 tok/s
MoE — speed based on ~3.85 GB active experts plus per-token routing overhead (full 25.02 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~60 tok/s
Q8_0 Context 256K Speed ~38 tok/s
Q4_K_M Context 128K Speed ~106 tok/s
MoE — speed based on ~1.65 GB active experts plus per-token routing overhead (full 17.05 GB loaded in VRAM)
Q8_0 Context 128K Speed ~79 tok/s
MoE — speed based on ~2.87 GB active experts plus per-token routing overhead (full 29.66 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~26 tok/s
Q8_0 Context 128K Speed ~15 tok/s
Q4_K_M Context 256K Speed ~91 tok/s
MoE — speed based on ~2.25 GB active experts plus per-token routing overhead (full 23.73 GB loaded in VRAM)
Q8_0 Context 256K Speed ~75 tok/s
MoE — speed based on ~3.1 GB active experts plus per-token routing overhead (full 32.6 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~105 tok/s
MoE — speed based on ~1.7 GB active experts plus per-token routing overhead (full 17.28 GB loaded in VRAM)
Q8_0 Context 256K Speed ~77 tok/s
MoE — speed based on ~2.98 GB active experts plus per-token routing overhead (full 30.25 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~26 tok/s
Q8_0 Context 128K Speed ~15 tok/s
Q4_K_M Context 128K Speed ~19 tok/s
Q8_0 Context 128K Speed ~11 tok/s
Q4_K_M Context 128K Speed ~40 tok/s
Q8_0 Context 128K Speed ~24 tok/s
Q4_K_M Context 256K Speed ~89 tok/s
MoE — speed based on ~2.3 GB active experts plus per-token routing overhead (full 23.03 GB loaded in VRAM)
Q8_0 Context 256K Speed ~75 tok/s
MoE — speed based on ~3.13 GB active experts plus per-token routing overhead (full 31.28 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~33 tok/s
MoE — speed based on ~9.49 GB active experts plus per-token routing overhead (full 60.87 GB loaded in VRAM)
Q8_0 Context 116K Speed ~20 tok/s
MoE — speed based on ~16.64 GB active experts plus per-token routing overhead (full 106.67 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~107 tok/s
Q8_0 Context 256K Speed ~73 tok/s
Q4_K_M Context 128K Speed ~66 tok/s
Q8_0 Context 128K Speed ~41 tok/s
Q4_K_M Context 128K Speed ~19 tok/s
Q8_0 Context 128K Speed ~11 tok/s
Q4_K_M Context 128K Speed ~23 tok/s
Q8_0 Context 128K Speed ~13 tok/s
Q4_K_M Context 256K Speed ~49 tok/s
Q8_0 Context 256K Speed ~29 tok/s
Q4_K_M Context 128K Speed ~47 tok/s
Q8_0 Context 128K Speed ~28 tok/s
Q4_K_M Context 128K Speed ~100 tok/s
MoE — speed based on ~1.86 GB active experts plus per-token routing overhead (full 10.83 GB loaded in VRAM)
Q8_0 Context 128K Speed ~98 tok/s
MoE — speed based on ~1.93 GB active experts plus per-token routing overhead (full 11.28 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~70 tok/s
Q8_0 Context 128K Speed ~44 tok/s
Q4_K_M Context 128K Speed ~70 tok/s
Q8_0 Context 128K Speed ~44 tok/s
Q4_K_M Context 128K Speed ~48 tok/s
Q8_0 Context 128K Speed ~30 tok/s
Q4_K_M Context 128K Speed ~40 tok/s
Q8_0 Context 128K Speed ~24 tok/s
Q4_K_M Context 256K Speed ~67 tok/s
MoE — speed based on ~3.71 GB active experts plus per-token routing overhead (full 111.39 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 175.4 GB
Q4_K_M Context 128K Speed ~54 tok/s
Q8_0 Context 128K Speed ~33 tok/s
Q4_K_M Context 128K Speed ~40 tok/s
Q8_0 Context 128K Speed ~24 tok/s
Q4_K_M Context 128K Speed ~26 tok/s
Q8_0 Context 128K Speed ~15 tok/s
Q4_K_M Context 128K Speed ~118 tok/s
Q8_0 Context 128K Speed ~79 tok/s
Q4_K_M Context 128K Speed ~115 tok/s
Q8_0 Context 128K Speed ~79 tok/s
Q4_K_M Context 128K Speed ~68 tok/s
Q8_0 Context 128K Speed ~42 tok/s
fp16 Context 128K Speed ~23 tok/s
Q4_K_M Context 128K Speed ~62 tok/s
Q8_0 Context 128K Speed ~44 tok/s
Q4_K_M Context 128K Speed ~47 tok/s
Q8_0 Context 128K Speed ~29 tok/s
Q4_K_M Context 128K Speed ~70 tok/s
Q8_0 Context 128K Speed ~44 tok/s
Q4_K_M Context 128K Speed ~136 tok/s
Q8_0 Context 128K Speed ~93 tok/s
Q4_K_M Context 32K Speed ~136 tok/s
Q8_0 Context 32K Speed ~93 tok/s
Q4_K_M Context 128K Speed ~62 tok/s
Q8_0 Context 128K Speed ~42 tok/s
Q4_K_M Context 32K Speed ~129 tok/s
Q8_0 Context 32K Speed ~88 tok/s
Q4_K_M Context 128K Speed ~133 tok/s
Q8_0 Context 128K Speed ~90 tok/s
fp16 Context 128K Speed ~54 tok/s
Q4_K_M Context 128K Speed ~115 tok/s
Q8_0 Context 128K Speed ~78 tok/s
Q4_K_M Context 32K Speed ~28 tok/s
Q8_0 Context 32K Speed ~16 tok/s
Q4_K_M Context 128K Speed ~90 tok/s
Q8_0 Context 128K Speed ~67 tok/s
Q4_K_M Context 128K Speed ~70 tok/s
Q8_0 Context 128K Speed ~44 tok/s
Q4_K_M Context 256K Speed ~176 tok/s
Q8_0 Context 256K Speed ~133 tok/s
Q4_K_M Context 16K Speed ~37 tok/s
Q8_0 Context 16K Speed ~22 tok/s
Q4_K_M Context 16K Speed ~79 tok/s
Q8_0 Context 16K Speed ~49 tok/s
Q4_K_M Context 128K Speed ~222 tok/s
Q8_0 Context 128K Speed ~172 tok/s
Q4_K_M Context 256K Speed ~260 tok/s
Q8_0 Context 256K Speed ~220 tok/s
Q4_K_M Context 1M Speed ~46 tok/s
MoE — speed based on ~6.3 GB active experts plus per-token routing overhead (full 65.29 GB loaded in VRAM)
Q8_0 Context 1M Speed ~28 tok/s
MoE — speed based on ~11.54 GB active experts plus per-token routing overhead (full 119.65 GB loaded in VRAM)
Q4_K_M Context 512K Speed ~61 tok/s
Q8_0 Context 512K Speed ~38 tok/s
Q4_K_M Context 512K Speed ~18 tok/s
Q8_0 Context 480K Speed ~11 tok/s
Q4_K_M Context 256K Speed ~101 tok/s
Q8_0 Context 256K Speed ~76 tok/s
Q4_K_M Context 256K Speed ~59 tok/s
Q8_0 Context 256K Speed ~37 tok/s
Q4_K_M Context 256K Speed ~105 tok/s
MoE — speed based on ~1.69 GB active experts plus per-token routing overhead (full 20.22 GB loaded in VRAM)
Q8_0 Context 256K Speed ~78 tok/s
MoE — speed based on ~2.94 GB active experts plus per-token routing overhead (full 35.21 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~104 tok/s
MoE — speed based on ~1.72 GB active experts plus per-token routing overhead (full 19.14 GB loaded in VRAM)
Q8_0 Context 256K Speed ~77 tok/s
MoE — speed based on ~2.97 GB active experts plus per-token routing overhead (full 33.15 GB loaded in VRAM)
Q8_0 Context 256K Speed ~35 tok/s
MoE — speed based on ~8.83 GB active experts plus per-token routing overhead (full 73.44 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~50 tok/s
MoE — speed based on ~5.65 GB active experts plus per-token routing overhead (full 70.64 GB loaded in VRAM)
Q8_0 Context 197K Speed ~32 tok/s
MoE — speed based on ~9.87 GB active experts plus per-token routing overhead (full 123.45 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~111 tok/s
MoE — speed based on ~1.48 GB active experts plus per-token routing overhead (full 23.61 GB loaded in VRAM)
Q8_0 Context 256K Speed ~86 tok/s
MoE — speed based on ~2.48 GB active experts plus per-token routing overhead (full 39.44 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~105 tok/s
MoE — speed based on ~1.68 GB active experts plus per-token routing overhead (full 19.71 GB loaded in VRAM)
Q8_0 Context 256K Speed ~78 tok/s
MoE — speed based on ~2.93 GB active experts plus per-token routing overhead (full 34.37 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~58 tok/s
Q8_0 Context 256K Speed ~38 tok/s
Q4_K_M Context 256K Speed ~191 tok/s
Q8_0 Context 256K Speed ~141 tok/s
Arete 27B 27.36B Basic Vision Tool use Reasoning MMLU
Q4_K_M Context 256K Speed ~22 tok/s
Q8_0 Context 256K Speed ~13 tok/s
Q4_K_S Context 140K Speed ~49 tok/s
MoE — speed based on ~5.77 GB active experts plus per-token routing overhead (full 103.84 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 195.2 GB
Q4_K_M Context 128K Speed ~23 tok/s
Q8_0 Context 128K Speed ~13 tok/s
Q4_K_M Context 128K Speed ~150 tok/s
Q8_0 Context 128K Speed ~103 tok/s
Q4_K_M Context 128K Speed ~115 tok/s
MoE — speed based on ~1.38 GB active experts plus per-token routing overhead (full 13.53 GB loaded in VRAM)
Q8_0 Context 128K Speed ~87 tok/s
MoE — speed based on ~2.43 GB active experts plus per-token routing overhead (full 23.74 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~109 tok/s
MoE — speed based on ~1.55 GB active experts plus per-token routing overhead (full 7.52 GB loaded in VRAM)
Q8_0 Context 128K Speed ~86 tok/s
MoE — speed based on ~2.48 GB active experts plus per-token routing overhead (full 12.04 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~191 tok/s
Q8_0 Context 128K Speed ~141 tok/s
Q4_K_M Context 64K Speed ~56 tok/s
Q8_0 Context 64K Speed ~44 tok/s
Tight fit 2 Q4_K_M Context 8K Speed ~23 tok/s
Only 8K context — under the 16K needed to be practical
Q8_0 Context 8K Speed ~13 tok/s
Only 8K context — under the 16K needed to be practical
Q4_K_M Context 8K Speed ~59 tok/s
Only 8K context — under the 16K needed to be practical
Q8_0 Context 8K Speed ~37 tok/s
Only 8K context — under the 16K needed to be practical
Notify me when new models fit my GPU An email when a new model fits your 128 GB setup — no spam, unsubscribe anytime.
Doesn't fit 20 Q4_K_M Needs 381.2 GB — 253.2 GB over
Q8_0 Needs 664.4 GB — 536.4 GB over
Q4_K_M Needs 586.5 GB — 458.5 GB over
Q8_0 Needs 1013.0 GB — 885.0 GB over
Q4_K_M Needs 128.9 GB — 0.9 GB over
Q8_0 Needs 226.5 GB — 98.5 GB over
View all 20 →
Doesn't fit 20 ✕ Q4_K_M Needs 381.2 GB — 253.2 GB over
Q8_0 Needs 664.4 GB — 536.4 GB over
Q4_K_M Needs 586.5 GB — 458.5 GB over
Q8_0 Needs 1013.0 GB — 885.0 GB over
Q4_K_M Needs 128.9 GB — 0.9 GB over
Q8_0 Needs 226.5 GB — 98.5 GB over
Q4_K_M Needs 900.1 GB — 772.1 GB over
Q8_0 Needs 1670.1 GB — 1542.1 GB over
Q4_K_M Needs 227.4 GB — 99.4 GB over
Q8_0 Needs 392.6 GB — 264.6 GB over
GLM-4.6 357B Excellent Reasoning Tool use MMLU 87
Q4_K_M Needs 200.8 GB — 72.8 GB over
Q8_0 Needs 353.3 GB — 225.3 GB over
Q4_K_M Needs 428.2 GB — 300.2 GB over
Q8_0 Needs 746.4 GB — 618.4 GB over
GLM 5.3 753.33B Excellent Tool use Reasoning MMLU 86.8
Q8_0 Needs 746.4 GB — 618.4 GB over
Q4_K_M Needs 161.1 GB — 33.1 GB over
Q8_0 Needs 282.1 GB — 154.1 GB over
Q4_K_M Needs 176.0 GB — 48.0 GB over
Q8_0 Needs 310.4 GB — 182.4 GB over
Q4_K_M Needs 226.3 GB — 98.3 GB over
Q8_0 Needs 396.8 GB — 268.8 GB over
GLM 5 753.86B Excellent Tool use MMLU 83
Q4_K_M Needs 424.7 GB — 296.7 GB over
Q8_0 Needs 746.4 GB — 618.4 GB over
Q4_K_M Needs 175.9 GB — 47.9 GB over
Q8_0 Needs 306.8 GB — 178.8 GB over
Q4_K_M Needs 129.1 GB — 1.1 GB over
Q8_0 Needs 226.7 GB — 98.7 GB over
Q4_K_M Needs 379.1 GB — 251.1 GB over
Q8_0 Needs 666.6 GB — 538.6 GB over
Q4_K_M Needs 591.3 GB — 463.3 GB over
Q8_0 Needs 1038.6 GB — 910.6 GB over
Q4_K_M Needs 465.3 GB — 337.3 GB over
Q8_0 Needs 811.9 GB — 683.9 GB over
Q4_K_M Needs 333.4 GB — 205.4 GB over
Q8_0 Needs 566.6 GB — 438.6 GB over
Q4_K_M Needs 293.0 GB — 165.0 GB over
Q8_0 Needs 496.2 GB — 368.2 GB over
Q4_K_M Needs 227.6 GB — 99.6 GB over
Q8_0 Needs 399.1 GB — 271.1 GB over