GPUs / By VRAM Best local LLMs for 24 GB VRAM (November 2026): what fits, context and speed With 24 GB of VRAM or unified memory, 70 models run well and 3 more are a tight fit. 34 don't fit.
Speeds assume 327 GB/s , the median memory bandwidth of the 16 GPUs and memory
configurations with exactly 24 GB. Yours may be faster or slower: pick your GPU for exact numbers.
Best overall Highest MMLU-Pro score among models that run well
Qwen3.6 27B Q4_K_M 134K context · ~19 tok/s · MMLU 86.2 Qwen3.5 27B Q4_K_M 137K context · ~18 tok/s · MMLU 86.1 Qwen3.5 35B A3B Q4_K_M 184K context · ~88 tok/s · MMLU 85.3 Gemma 4 31B Q4_K_M 91K context · ~17 tok/s · MMLU 85.2 Qwen3.6 35B A3B Q4_K_M 256K context · ~88 tok/s · MMLU 85.2 Best for coding Highest SWE-bench Verified score among models that run well
Qwen3.8 27B Q4_K_M 142K context · ~19 tok/s · SWE-bench 86 Ornith 1.5 35B A3B Q4_K_M 198K context · ~90 tok/s · SWE-bench 79 Qwen3.6 27B Q4_K_M 134K context · ~19 tok/s · SWE-bench 77.2 Muse Glimmer 30B Q4_K_M 128K context · ~18 tok/s · SWE-bench 76 Qwen3.6 35B A3B Q4_K_M 256K context · ~88 tok/s · SWE-bench 73.4 GPUs with 24 GB of memory Unified memory (Apple Silicon, Ryzen AI Max, RTX Spark) Upgrade options 32 GB runs 1 more
model well.
The cheapest GPUs with more than 24 GB, by launch MSRP.
As an Amazon Associate, I earn from qualifying purchases.
Every model in 24 GB 123 of 214 model configurations can run on 24 GB — sorted by quality
107 models, each counted once by the best way it runs on your hardware.
Fastest good-quality model Phi-3 Mini 3.8B ~96 tok/s · Q4_K_M · 59K context Runs well 70 Q4_K_M Context 134K Speed ~19 tok/s
Q8_0 Doesn't fit · needs 28.1 GB
Q4_K_M Context 137K Speed ~18 tok/s
Q8_0 Doesn't fit · needs 26.7 GB
Q4_K_M Context 184K Speed ~88 tok/s
MoE — speed based on ~1.76 GB active experts plus per-token routing overhead (full 20.5 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 34.4 GB
View all 70 →
Runs well 70 ✕ Q4_K_M Context 134K Speed ~19 tok/s
Q8_0 Doesn't fit · needs 28.1 GB
Q4_K_M Context 137K Speed ~18 tok/s
Q8_0 Doesn't fit · needs 26.7 GB
Q4_K_M Context 184K Speed ~88 tok/s
MoE — speed based on ~1.76 GB active experts plus per-token routing overhead (full 20.5 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 34.4 GB
Q4_K_M Context 256K Speed ~88 tok/s
MoE — speed based on ~1.77 GB active experts plus per-token routing overhead (full 20.61 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 34.4 GB
Q4_K_M Context 91K Speed ~17 tok/s
Q8_0 Doesn't fit · needs 30.5 GB
Q4_K_M Context 22K Speed ~16 tok/s
Q8_0 Doesn't fit · needs 32.7 GB
Q4_K_M Context 16K Speed ~33 tok/s
Q8_0 Context 16K Speed ~20 tok/s
Q4_K_M Context 142K Speed ~19 tok/s
Q8_0 Doesn't fit · needs 27.1 GB
Q4_K_M Context 22K Speed ~16 tok/s
Q8_0 Doesn't fit · needs 32.7 GB
QwQ 32B 32.5B Excellent Reasoning Tool use MMLU 83
Q4_K_M Context 22K Speed ~16 tok/s
Q8_0 Doesn't fit · needs 32.7 GB
Q4_K_M Context 256K Speed ~74 tok/s
MoE — speed based on ~2.44 GB active experts plus per-token routing overhead (full 15.87 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 25.0 GB
Q4_K_M Context 256K Speed ~49 tok/s
Q8_0 Context 256K Speed ~31 tok/s
Q4_K_M Context 128K Speed ~91 tok/s
MoE — speed based on ~1.65 GB active experts plus per-token routing overhead (full 17.05 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 29.7 GB
Q4_K_M Context 69K Speed ~21 tok/s
Q8_0 Tight fit · 4K context
Q4_K_M Context 44K Speed ~77 tok/s
MoE — speed based on ~2.25 GB active experts plus per-token routing overhead (full 23.73 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 32.6 GB
Q4_K_M Context 73K Speed ~90 tok/s
MoE — speed based on ~1.7 GB active experts plus per-token routing overhead (full 17.28 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 30.3 GB
Q4_K_M Context 69K Speed ~21 tok/s
Q8_0 Tight fit · 4K context
Q4_K_M Context 22K Speed ~16 tok/s
Q8_0 Doesn't fit · needs 32.7 GB
Q4_K_M Context 85K Speed ~33 tok/s
Q8_0 Context 51K Speed ~20 tok/s
Q4_K_M Context 161K Speed ~76 tok/s
MoE — speed based on ~2.3 GB active experts plus per-token routing overhead (full 23.03 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 31.3 GB
Q4_K_M Context 256K Speed ~87 tok/s
Q8_0 Context 256K Speed ~60 tok/s
Q4_K_M Context 128K Speed ~54 tok/s
Q8_0 Context 115K Speed ~34 tok/s
Q4_K_M Context 22K Speed ~16 tok/s
Q8_0 Doesn't fit · needs 32.7 GB
Q4_K_M Context 113K Speed ~19 tok/s
Q8_0 Doesn't fit · needs 26.8 GB
Q4_K_M Context 256K Speed ~40 tok/s
Q8_0 Context 256K Speed ~24 tok/s
Q4_K_M Context 128K Speed ~38 tok/s
Q8_0 Context 128K Speed ~23 tok/s
Q4_K_M Context 128K Speed ~86 tok/s
MoE — speed based on ~1.86 GB active experts plus per-token routing overhead (full 10.83 GB loaded in VRAM)
Q8_0 Context 128K Speed ~84 tok/s
MoE — speed based on ~1.93 GB active experts plus per-token routing overhead (full 11.28 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~58 tok/s
Q8_0 Context 128K Speed ~36 tok/s
Q4_K_M Context 128K Speed ~58 tok/s
Q8_0 Context 128K Speed ~36 tok/s
Q4_K_M Context 128K Speed ~39 tok/s
Q8_0 Context 128K Speed ~24 tok/s
Q4_K_M Context 85K Speed ~33 tok/s
Q8_0 Context 51K Speed ~20 tok/s
Q4_K_M Context 128K Speed ~44 tok/s
Q8_0 Context 111K Speed ~27 tok/s
Q4_K_M Context 85K Speed ~33 tok/s
Q8_0 Context 51K Speed ~20 tok/s
Q4_K_M Context 69K Speed ~21 tok/s
Q8_0 Tight fit · 4K context
Q4_K_M Context 59K Speed ~96 tok/s
Q8_0 Context 55K Speed ~65 tok/s
Q4_K_M Context 128K Speed ~94 tok/s
Q8_0 Context 128K Speed ~64 tok/s
Q4_K_M Context 128K Speed ~55 tok/s
Q8_0 Context 128K Speed ~34 tok/s
fp16 Context 74K Speed ~19 tok/s
Q4_K_M Context 128K Speed ~51 tok/s
Q8_0 Context 128K Speed ~36 tok/s
Q4_K_M Context 128K Speed ~58 tok/s
Q8_0 Context 128K Speed ~36 tok/s
Q4_K_M Context 89K Speed ~39 tok/s
Q8_0 Context 62K Speed ~23 tok/s
Q4_K_M Context 128K Speed ~112 tok/s
Q8_0 Context 128K Speed ~76 tok/s
Q4_K_M Context 32K Speed ~112 tok/s
Q8_0 Context 32K Speed ~76 tok/s
Q4_K_M Context 128K Speed ~51 tok/s
Q8_0 Context 117K Speed ~34 tok/s
Q4_K_M Context 32K Speed ~105 tok/s
Q8_0 Context 32K Speed ~72 tok/s
Q4_K_M Context 128K Speed ~108 tok/s
Q8_0 Context 128K Speed ~74 tok/s
fp16 Context 128K Speed ~44 tok/s
Q4_K_M Context 128K Speed ~94 tok/s
Q8_0 Context 128K Speed ~64 tok/s
Q4_K_M Context 32K Speed ~23 tok/s
Q8_0 Tight fit · 9K context
Q4_K_M Context 128K Speed ~73 tok/s
Q8_0 Context 128K Speed ~55 tok/s
Q4_K_M Context 128K Speed ~58 tok/s
Q8_0 Context 128K Speed ~36 tok/s
Q4_K_M Context 256K Speed ~144 tok/s
Q8_0 Context 256K Speed ~109 tok/s
Q4_K_M Context 16K Speed ~30 tok/s
Q8_0 Context 16K Speed ~18 tok/s
Q4_K_M Context 16K Speed ~64 tok/s
Q8_0 Context 16K Speed ~40 tok/s
Q4_K_M Context 128K Speed ~181 tok/s
Q8_0 Context 128K Speed ~141 tok/s
Q4_K_M Context 256K Speed ~213 tok/s
Q8_0 Context 256K Speed ~180 tok/s
Q4_K_M Context 256K Speed ~82 tok/s
Q8_0 Context 256K Speed ~63 tok/s
Q4_K_M Context 256K Speed ~48 tok/s
Q8_0 Context 256K Speed ~30 tok/s
Q4_K_M Context 256K Speed ~48 tok/s
Q8_0 Context 256K Speed ~31 tok/s
Q4_K_M Context 256K Speed ~156 tok/s
Q8_0 Context 256K Speed ~115 tok/s
Q4_K_M Context 225K Speed ~90 tok/s
MoE — speed based on ~1.68 GB active experts plus per-token routing overhead (full 19.71 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 34.4 GB
Q4_K_M Context 198K Speed ~90 tok/s
MoE — speed based on ~1.69 GB active experts plus per-token routing overhead (full 20.22 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 35.2 GB
Q4_K_M Context 137K Speed ~50 tok/s
Q8_0 Context 110K Speed ~31 tok/s
Arete 27B 27.36B Basic Vision Tool use Reasoning MMLU
Q4_K_M Context 130K Speed ~18 tok/s
Q8_0 Doesn't fit · needs 26.8 GB
Q4_K_M Context 128K Speed ~18 tok/s
Q8_0 Doesn't fit · needs 27.6 GB
Q4_K_M Context 128K Speed ~122 tok/s
Q8_0 Context 128K Speed ~84 tok/s
Q4_K_M Context 128K Speed ~99 tok/s
MoE — speed based on ~1.38 GB active experts plus per-token routing overhead (full 13.53 GB loaded in VRAM)
Q8_0 Tight fit · 13K context
Q4_K_M Context 128K Speed ~94 tok/s
MoE — speed based on ~1.55 GB active experts plus per-token routing overhead (full 7.52 GB loaded in VRAM)
Q8_0 Context 128K Speed ~73 tok/s
MoE — speed based on ~2.48 GB active experts plus per-token routing overhead (full 12.04 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~156 tok/s
Q8_0 Context 128K Speed ~115 tok/s
Q4_K_M Context 127K Speed ~89 tok/s
MoE — speed based on ~1.72 GB active experts plus per-token routing overhead (full 19.14 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 33.2 GB
Q4_K_M Context 64K Speed ~46 tok/s
Q8_0 Context 64K Speed ~36 tok/s
Q4_K_M Context 23K Speed ~15 tok/s
Q8_0 Doesn't fit · needs 33.0 GB
Notify me when new models fit my GPU An email when a new model fits your 24 GB setup — no spam, unsubscribe anytime.
Doesn't fit 34 Q4_K_M Needs 381.2 GB — 357.2 GB over
Q8_0 Needs 664.4 GB — 640.4 GB over
Q4_K_M Needs 45.1 GB — 21.1 GB over
Q8_0 Needs 79.0 GB — 55.0 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 586.5 GB — 562.5 GB over
Q8_0 Needs 1013.0 GB — 989.0 GB over
View all 34 →
Doesn't fit 34 ✕ Q4_K_M Needs 381.2 GB — 357.2 GB over
Q8_0 Needs 664.4 GB — 640.4 GB over
Q4_K_M Needs 45.1 GB — 21.1 GB over
Q8_0 Needs 79.0 GB — 55.0 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 586.5 GB — 562.5 GB over
Q8_0 Needs 1013.0 GB — 989.0 GB over
Q4_K_M Needs 68.2 GB — 44.2 GB over
Q8_0 Needs 118.3 GB — 94.3 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 128.9 GB — 104.9 GB over
Q8_0 Needs 226.5 GB — 202.5 GB over
Q4_K_M Needs 900.1 GB — 876.1 GB over
Q8_0 Needs 1670.1 GB — 1646.1 GB over
Q4_K_M Needs 227.4 GB — 203.4 GB over
Q8_0 Needs 392.6 GB — 368.6 GB over
GLM-4.6 357B Excellent Reasoning Tool use MMLU 87
Q4_K_M Needs 200.8 GB — 176.8 GB over
Q8_0 Needs 353.3 GB — 329.3 GB over
Q4_K_M Needs 428.2 GB — 404.2 GB over
Q8_0 Needs 746.4 GB — 722.4 GB over
GLM 5.3 753.33B Excellent Tool use Reasoning MMLU 86.8
Q8_0 Needs 746.4 GB — 722.4 GB over
Q4_K_M Needs 161.1 GB — 137.1 GB over
Q8_0 Needs 282.1 GB — 258.1 GB over
Q4_K_M Needs 176.0 GB — 152.0 GB over
Q8_0 Needs 310.4 GB — 286.4 GB over
Q4_K_M Needs 44.5 GB — 20.5 GB over
Q8_0 Needs 72.3 GB — 48.3 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 44.5 GB — 20.5 GB over
Q8_0 Needs 72.3 GB — 48.3 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 39.9 GB — 15.9 GB over
Q8_0 Needs 70.1 GB — 46.1 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 226.3 GB — 202.3 GB over
Q8_0 Needs 396.8 GB — 372.8 GB over
Q4_K_M Needs 71.3 GB — 47.3 GB over
Q8_0 Needs 121.0 GB — 97.0 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 111.9 GB — 87.9 GB over
Q8_0 Needs 195.2 GB — 171.2 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 39.9 GB — 15.9 GB over
Q8_0 Needs 70.1 GB — 46.1 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
GLM 5 753.86B Excellent Tool use MMLU 83
Q4_K_M Needs 424.7 GB — 400.7 GB over
Q8_0 Needs 746.4 GB — 722.4 GB over
Q4_K_M Needs 175.9 GB — 151.9 GB over
Q8_0 Needs 306.8 GB — 282.8 GB over
Q4_K_M Needs 129.1 GB — 105.1 GB over
Q8_0 Needs 226.7 GB — 202.7 GB over
Q4_K_M Needs 61.1 GB — 37.1 GB over
Q8_0 Needs 106.9 GB — 82.9 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 111.4 GB — 87.4 GB over
Q8_0 Needs 175.4 GB — 151.4 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 379.1 GB — 355.1 GB over
Q8_0 Needs 666.6 GB — 642.6 GB over
Q4_K_M Needs 591.3 GB — 567.3 GB over
Q8_0 Needs 1038.6 GB — 1014.6 GB over
Q4_K_M Needs 465.3 GB — 441.3 GB over
Q8_0 Needs 811.9 GB — 787.9 GB over
Q4_K_M Needs 333.4 GB — 309.4 GB over
Q8_0 Needs 566.6 GB — 542.6 GB over
Q4_K_M Needs 293.0 GB — 269.0 GB over
Q8_0 Needs 496.2 GB — 472.2 GB over
Q4_K_M Needs 227.6 GB — 203.6 GB over
Q8_0 Needs 399.1 GB — 375.1 GB over
Q4_K_S Needs 104.0 GB — 80.0 GB over
Q8_0 Needs 195.2 GB — 171.2 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 70.7 GB — 46.7 GB over
Q8_0 Needs 123.5 GB — 99.5 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 65.3 GB — 41.3 GB over
Q8_0 Needs 119.7 GB — 95.7 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q8_0 Needs 73.4 GB — 49.4 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.