GPUs / By VRAM Best local LLMs for 96 GB VRAM (November 2026): what fits, context and speed With 96 GB of VRAM or unified memory, 82 models run well and 2 more are a tight fit. 23 don't fit.
Speeds assume 487 GB/s , the median memory bandwidth of the 6 GPUs and memory
configurations with exactly 96 GB. Yours may be faster or slower: pick your GPU for exact numbers.
Best for coding Highest SWE-bench Verified score among models that run well
Qwen3.8 27B Q8_0 256K context · ~16 tok/s · SWE-bench 86 Ornith 1.5 122B Q4_K_M 256K context · ~59 tok/s · SWE-bench 84.5 Ornith 1.5 35B A3B Q8_0 256K context · ~91 tok/s · SWE-bench 79 Qwen3.6 27B Q8_0 256K context · ~16 tok/s · SWE-bench 77.2 Muse Glimmer 30B Q8_0 128K context · ~16 tok/s · SWE-bench 76 GPUs with 96 GB of memory Unified memory (Apple Silicon, Ryzen AI Max, RTX Spark) Upgrade options 128 GB runs 3 more
models well, including Step 3.5 Flash and Qwen3.8 Flash Next .
The cheapest GPUs with more than 96 GB, by launch MSRP.
As an Amazon Associate, I earn from qualifying purchases.
Every model in 96 GB 164 of 214 model configurations can run on 96 GB — sorted by quality
107 models, each counted once by the best way it runs on your hardware.
Largest that runs well Ornith 1.5 122B 125.09B params · Q4_K_M · 256K context Fastest good-quality model Phi-3 Mini 3.8B ~143 tok/s · Q4_K_M · 128K context Runs well 82 View all 82 →
Runs well 82 ✕ Q4_K_M Context 256K Speed ~121 tok/s
MoE — speed based on ~1.69 GB active experts plus per-token routing overhead (full 45.09 GB loaded in VRAM)
Q8_0 Context 256K Speed ~90 tok/s
MoE — speed based on ~2.96 GB active experts plus per-token routing overhead (full 78.99 GB loaded in VRAM)
Q4_K_M Context 51K Speed ~79 tok/s
MoE — speed based on ~3.7 GB active experts plus per-token routing overhead (full 67.65 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 118.3 GB
Q4_K_M Context 256K Speed ~28 tok/s
Q8_0 Context 256K Speed ~16 tok/s
Q4_K_M Context 256K Speed ~27 tok/s
Q8_0 Context 256K Speed ~16 tok/s
Q4_K_M Context 128K Speed ~10 tok/s
Q8_0 Context 78K Speed ~6 tok/s
Q4_K_M Context 128K Speed ~10 tok/s
Q8_0 Context 78K Speed ~6 tok/s
Q4_K_M Context 128K Speed ~11 tok/s
Q8_0 Context 85K Speed ~6 tok/s
Q4_K_M Context 256K Speed ~119 tok/s
MoE — speed based on ~1.76 GB active experts plus per-token routing overhead (full 20.5 GB loaded in VRAM)
Q8_0 Context 256K Speed ~90 tok/s
MoE — speed based on ~2.95 GB active experts plus per-token routing overhead (full 34.37 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~25 tok/s
Q8_0 Context 256K Speed ~14 tok/s
Q4_K_M Context 256K Speed ~118 tok/s
MoE — speed based on ~1.77 GB active experts plus per-token routing overhead (full 20.61 GB loaded in VRAM)
Q8_0 Context 256K Speed ~90 tok/s
MoE — speed based on ~2.95 GB active experts plus per-token routing overhead (full 34.37 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~57 tok/s
MoE — speed based on ~5.84 GB active experts plus per-token routing overhead (full 71.28 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 121.0 GB
Q4_K_M Context 128K Speed ~23 tok/s
Q8_0 Context 128K Speed ~14 tok/s
Q4_K_M Context 16K Speed ~48 tok/s
Q8_0 Context 16K Speed ~29 tok/s
Q4_K_M Context 256K Speed ~28 tok/s
Q8_0 Context 256K Speed ~16 tok/s
Q4_K_M Context 128K Speed ~11 tok/s
Q8_0 Context 85K Speed ~6 tok/s
Q4_K_M Context 128K Speed ~23 tok/s
Q8_0 Context 128K Speed ~14 tok/s
QwQ 32B 32.5B Excellent Reasoning Tool use MMLU 83
Q4_K_M Context 128K Speed ~23 tok/s
Q8_0 Context 128K Speed ~14 tok/s
Q4_K_M Context 256K Speed ~101 tok/s
MoE — speed based on ~2.44 GB active experts plus per-token routing overhead (full 15.87 GB loaded in VRAM)
Q8_0 Context 256K Speed ~77 tok/s
MoE — speed based on ~3.85 GB active experts plus per-token routing overhead (full 25.02 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~73 tok/s
Q8_0 Context 256K Speed ~46 tok/s
Q4_K_M Context 128K Speed ~122 tok/s
MoE — speed based on ~1.65 GB active experts plus per-token routing overhead (full 17.05 GB loaded in VRAM)
Q8_0 Context 128K Speed ~92 tok/s
MoE — speed based on ~2.87 GB active experts plus per-token routing overhead (full 29.66 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~32 tok/s
Q8_0 Context 128K Speed ~19 tok/s
Q4_K_M Context 256K Speed ~105 tok/s
MoE — speed based on ~2.25 GB active experts plus per-token routing overhead (full 23.73 GB loaded in VRAM)
Q8_0 Context 256K Speed ~88 tok/s
MoE — speed based on ~3.1 GB active experts plus per-token routing overhead (full 32.6 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~120 tok/s
MoE — speed based on ~1.7 GB active experts plus per-token routing overhead (full 17.28 GB loaded in VRAM)
Q8_0 Context 256K Speed ~90 tok/s
MoE — speed based on ~2.98 GB active experts plus per-token routing overhead (full 30.25 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~32 tok/s
Q8_0 Context 128K Speed ~19 tok/s
Q4_K_M Context 128K Speed ~23 tok/s
Q8_0 Context 128K Speed ~14 tok/s
Q4_K_M Context 128K Speed ~49 tok/s
Q8_0 Context 128K Speed ~29 tok/s
Q4_K_M Context 256K Speed ~104 tok/s
MoE — speed based on ~2.3 GB active experts plus per-token routing overhead (full 23.03 GB loaded in VRAM)
Q8_0 Context 256K Speed ~87 tok/s
MoE — speed based on ~3.13 GB active experts plus per-token routing overhead (full 31.28 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~39 tok/s
MoE — speed based on ~9.49 GB active experts plus per-token routing overhead (full 60.87 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 106.9 GB
Q4_K_M Context 256K Speed ~130 tok/s
Q8_0 Context 256K Speed ~89 tok/s
Q4_K_M Context 128K Speed ~81 tok/s
Q8_0 Context 128K Speed ~50 tok/s
Q4_K_M Context 128K Speed ~23 tok/s
Q8_0 Context 128K Speed ~14 tok/s
Q4_K_M Context 128K Speed ~28 tok/s
Q8_0 Context 128K Speed ~16 tok/s
Q4_K_M Context 256K Speed ~60 tok/s
Q8_0 Context 256K Speed ~36 tok/s
Q4_K_M Context 128K Speed ~57 tok/s
Q8_0 Context 128K Speed ~35 tok/s
Q4_K_M Context 128K Speed ~116 tok/s
MoE — speed based on ~1.86 GB active experts plus per-token routing overhead (full 10.83 GB loaded in VRAM)
Q8_0 Context 128K Speed ~114 tok/s
MoE — speed based on ~1.93 GB active experts plus per-token routing overhead (full 11.28 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~86 tok/s
Q8_0 Context 128K Speed ~54 tok/s
Q4_K_M Context 128K Speed ~86 tok/s
Q8_0 Context 128K Speed ~54 tok/s
Q4_K_M Context 128K Speed ~59 tok/s
Q8_0 Context 128K Speed ~36 tok/s
Q4_K_M Context 128K Speed ~49 tok/s
Q8_0 Context 128K Speed ~29 tok/s
Q4_K_M Context 128K Speed ~66 tok/s
Q8_0 Context 128K Speed ~40 tok/s
Q4_K_M Context 128K Speed ~49 tok/s
Q8_0 Context 128K Speed ~29 tok/s
Q4_K_M Context 128K Speed ~32 tok/s
Q8_0 Context 128K Speed ~19 tok/s
Q4_K_M Context 128K Speed ~143 tok/s
Q8_0 Context 128K Speed ~96 tok/s
Q4_K_M Context 128K Speed ~139 tok/s
Q8_0 Context 128K Speed ~96 tok/s
Q4_K_M Context 128K Speed ~82 tok/s
Q8_0 Context 128K Speed ~51 tok/s
fp16 Context 128K Speed ~29 tok/s
Q4_K_M Context 128K Speed ~76 tok/s
Q8_0 Context 128K Speed ~54 tok/s
Q4_K_M Context 128K Speed ~57 tok/s
Q8_0 Context 128K Speed ~35 tok/s
Q4_K_M Context 128K Speed ~86 tok/s
Q8_0 Context 128K Speed ~54 tok/s
Q4_K_M Context 128K Speed ~166 tok/s
Q8_0 Context 128K Speed ~114 tok/s
Q4_K_M Context 32K Speed ~166 tok/s
Q8_0 Context 32K Speed ~114 tok/s
Q4_K_M Context 128K Speed ~76 tok/s
Q8_0 Context 128K Speed ~51 tok/s
Q4_K_M Context 32K Speed ~157 tok/s
Q8_0 Context 32K Speed ~107 tok/s
Q4_K_M Context 128K Speed ~161 tok/s
Q8_0 Context 128K Speed ~110 tok/s
fp16 Context 128K Speed ~66 tok/s
Q4_K_M Context 128K Speed ~139 tok/s
Q8_0 Context 128K Speed ~95 tok/s
Q4_K_M Context 32K Speed ~34 tok/s
Q8_0 Context 32K Speed ~20 tok/s
Q4_K_M Context 128K Speed ~109 tok/s
Q8_0 Context 128K Speed ~82 tok/s
Q4_K_M Context 128K Speed ~86 tok/s
Q8_0 Context 128K Speed ~54 tok/s
Q4_K_M Context 256K Speed ~214 tok/s
Q8_0 Context 256K Speed ~162 tok/s
Q4_K_M Context 16K Speed ~45 tok/s
Q8_0 Context 16K Speed ~27 tok/s
Q4_K_M Context 16K Speed ~96 tok/s
Q8_0 Context 16K Speed ~60 tok/s
Q4_K_M Context 128K Speed ~270 tok/s
Q8_0 Context 128K Speed ~210 tok/s
Q4_K_M Context 256K Speed ~317 tok/s
Q8_0 Context 256K Speed ~268 tok/s
Q4_K_M Context 1M Speed ~54 tok/s
MoE — speed based on ~6.3 GB active experts plus per-token routing overhead (full 65.29 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 119.7 GB
Q4_K_M Context 512K Speed ~74 tok/s
Q8_0 Context 512K Speed ~46 tok/s
Q4_K_M Context 387K Speed ~22 tok/s
Q8_0 Context 319K Speed ~13 tok/s
Q4_K_M Context 256K Speed ~123 tok/s
Q8_0 Context 256K Speed ~93 tok/s
Q4_K_M Context 256K Speed ~72 tok/s
Q8_0 Context 256K Speed ~45 tok/s
Q4_K_M Context 256K Speed ~121 tok/s
MoE — speed based on ~1.69 GB active experts plus per-token routing overhead (full 20.22 GB loaded in VRAM)
Q8_0 Context 256K Speed ~91 tok/s
MoE — speed based on ~2.94 GB active experts plus per-token routing overhead (full 35.21 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~120 tok/s
MoE — speed based on ~1.72 GB active experts plus per-token routing overhead (full 19.14 GB loaded in VRAM)
Q8_0 Context 256K Speed ~90 tok/s
MoE — speed based on ~2.97 GB active experts plus per-token routing overhead (full 33.15 GB loaded in VRAM)
Q8_0 Context 256K Speed ~42 tok/s
MoE — speed based on ~8.83 GB active experts plus per-token routing overhead (full 73.44 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~59 tok/s
MoE — speed based on ~5.65 GB active experts plus per-token routing overhead (full 70.64 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 123.5 GB
Q4_K_M Context 256K Speed ~128 tok/s
MoE — speed based on ~1.48 GB active experts plus per-token routing overhead (full 23.61 GB loaded in VRAM)
Q8_0 Context 256K Speed ~100 tok/s
MoE — speed based on ~2.48 GB active experts plus per-token routing overhead (full 39.44 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~121 tok/s
MoE — speed based on ~1.68 GB active experts plus per-token routing overhead (full 19.71 GB loaded in VRAM)
Q8_0 Context 256K Speed ~91 tok/s
MoE — speed based on ~2.93 GB active experts plus per-token routing overhead (full 34.37 GB loaded in VRAM)
Q4_K_M Context 256K Speed ~71 tok/s
Q8_0 Context 256K Speed ~46 tok/s
Q4_K_M Context 256K Speed ~232 tok/s
Q8_0 Context 256K Speed ~171 tok/s
Arete 27B 27.36B Basic Vision Tool use Reasoning MMLU
Q4_K_M Context 256K Speed ~27 tok/s
Q8_0 Context 256K Speed ~16 tok/s
Q4_K_M Context 128K Speed ~27 tok/s
Q8_0 Context 128K Speed ~16 tok/s
Q4_K_M Context 128K Speed ~182 tok/s
Q8_0 Context 128K Speed ~126 tok/s
Q4_K_M Context 128K Speed ~132 tok/s
MoE — speed based on ~1.38 GB active experts plus per-token routing overhead (full 13.53 GB loaded in VRAM)
Q8_0 Context 128K Speed ~101 tok/s
MoE — speed based on ~2.43 GB active experts plus per-token routing overhead (full 23.74 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~125 tok/s
MoE — speed based on ~1.55 GB active experts plus per-token routing overhead (full 7.52 GB loaded in VRAM)
Q8_0 Context 128K Speed ~100 tok/s
MoE — speed based on ~2.48 GB active experts plus per-token routing overhead (full 12.04 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~232 tok/s
Q8_0 Context 128K Speed ~171 tok/s
Q4_K_M Context 64K Speed ~68 tok/s
Q8_0 Context 64K Speed ~54 tok/s
Tight fit 2 Q4_K_M Context 8K Speed ~28 tok/s
Only 8K context — under the 16K needed to be practical
Q8_0 Context 8K Speed ~16 tok/s
Only 8K context — under the 16K needed to be practical
Q4_K_M Context 8K Speed ~72 tok/s
Only 8K context — under the 16K needed to be practical
Q8_0 Context 8K Speed ~45 tok/s
Only 8K context — under the 16K needed to be practical
Notify me when new models fit my GPU An email when a new model fits your 96 GB setup — no spam, unsubscribe anytime.
Doesn't fit 23 Q4_K_M Needs 381.2 GB — 285.2 GB over
Q8_0 Needs 664.4 GB — 568.4 GB over
Q4_K_M Needs 586.5 GB — 490.5 GB over
Q8_0 Needs 1013.0 GB — 917.0 GB over
Q4_K_M Needs 128.9 GB — 32.9 GB over
Q8_0 Needs 226.5 GB — 130.5 GB over
View all 23 →
Doesn't fit 23 ✕ Q4_K_M Needs 381.2 GB — 285.2 GB over
Q8_0 Needs 664.4 GB — 568.4 GB over
Q4_K_M Needs 586.5 GB — 490.5 GB over
Q8_0 Needs 1013.0 GB — 917.0 GB over
Q4_K_M Needs 128.9 GB — 32.9 GB over
Q8_0 Needs 226.5 GB — 130.5 GB over
Q4_K_M Needs 900.1 GB — 804.1 GB over
Q8_0 Needs 1670.1 GB — 1574.1 GB over
Q4_K_M Needs 227.4 GB — 131.4 GB over
Q8_0 Needs 392.6 GB — 296.6 GB over
GLM-4.6 357B Excellent Reasoning Tool use MMLU 87
Q4_K_M Needs 200.8 GB — 104.8 GB over
Q8_0 Needs 353.3 GB — 257.3 GB over
Q4_K_M Needs 428.2 GB — 332.2 GB over
Q8_0 Needs 746.4 GB — 650.4 GB over
GLM 5.3 753.33B Excellent Tool use Reasoning MMLU 86.8
Q8_0 Needs 746.4 GB — 650.4 GB over
Q4_K_M Needs 161.1 GB — 65.1 GB over
Q8_0 Needs 282.1 GB — 186.1 GB over
Q4_K_M Needs 176.0 GB — 80.0 GB over
Q8_0 Needs 310.4 GB — 214.4 GB over
Q4_K_M Needs 226.3 GB — 130.3 GB over
Q8_0 Needs 396.8 GB — 300.8 GB over
Q4_K_M Needs 111.9 GB — 15.9 GB over
Q8_0 Needs 195.2 GB — 99.2 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
GLM 5 753.86B Excellent Tool use MMLU 83
Q4_K_M Needs 424.7 GB — 328.7 GB over
Q8_0 Needs 746.4 GB — 650.4 GB over
Q4_K_M Needs 175.9 GB — 79.9 GB over
Q8_0 Needs 306.8 GB — 210.8 GB over
Q4_K_M Needs 129.1 GB — 33.1 GB over
Q8_0 Needs 226.7 GB — 130.7 GB over
Q4_K_M Needs 111.4 GB — 15.4 GB over
Q8_0 Needs 175.4 GB — 79.4 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 379.1 GB — 283.1 GB over
Q8_0 Needs 666.6 GB — 570.6 GB over
Q4_K_M Needs 591.3 GB — 495.3 GB over
Q8_0 Needs 1038.6 GB — 942.6 GB over
Q4_K_M Needs 465.3 GB — 369.3 GB over
Q8_0 Needs 811.9 GB — 715.9 GB over
Q4_K_M Needs 333.4 GB — 237.4 GB over
Q8_0 Needs 566.6 GB — 470.6 GB over
Q4_K_M Needs 293.0 GB — 197.0 GB over
Q8_0 Needs 496.2 GB — 400.2 GB over
Q4_K_M Needs 227.6 GB — 131.6 GB over
Q8_0 Needs 399.1 GB — 303.1 GB over
Q4_K_S Needs 104.0 GB — 8.0 GB over
Q8_0 Needs 195.2 GB — 99.2 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.