GPUs / By VRAM Best local LLMs for 12 GB VRAM (November 2026): what fits, context and speed With 12 GB of VRAM or unified memory, 42 models run well and 1 more are a tight fit. 64 don't fit.
Speeds assume 444 GB/s , the median memory bandwidth of the 14 GPUs and memory
configurations with exactly 12 GB. Yours may be faster or slower: pick your GPU for exact numbers.
Best overall Highest MMLU-Pro score among models that run well
Phi-4 14B Q4_K_M 16K context · ~44 tok/s · MMLU 84.8 Qwen3.5 9B Q8_0 100K context · ~42 tok/s · MMLU 82.5 Qwen 2.5 14B Q4_K_M 19K context · ~44 tok/s · MMLU 79.9 Qwen3.5 4B Q8_0 252K context · ~81 tok/s · MMLU 79.1 Qwen3 8B Q8_0 28K context · ~46 tok/s · MMLU 79 GPUs with 12 GB of memory Every model in 12 GB 79 of 214 model configurations can run on 12 GB — sorted by quality
107 models, each counted once by the best way it runs on your hardware.
Largest that runs well GPT-OSS 20B 21B params · Q8_0 · 31K context Fastest good-quality model Phi-3 Mini 3.8B ~131 tok/s · Q4_K_M · 26K context Runs well 42 View all 42 →
Runs well 42 ✕ Q4_K_M Context 16K Speed ~44 tok/s
Q8_0 Doesn't fit · needs 14.7 GB
Q4_K_M Context 216K Speed ~66 tok/s
Q8_0 Context 100K Speed ~42 tok/s
Q4_K_M Context 19K Speed ~44 tok/s
Q8_0 Doesn't fit · needs 14.8 GB
Q4_K_M Context 256K Speed ~119 tok/s
Q8_0 Context 252K Speed ~81 tok/s
Q4_K_M Context 53K Speed ~74 tok/s
Q8_0 Context 28K Speed ~46 tok/s
Q4_K_M Context 256K Speed ~55 tok/s
Q8_0 Tight fit · 13K context
Q4_K_M Context 128K Speed ~52 tok/s
Q8_0 Doesn't fit · needs 12.2 GB
Q4_K_M Context 50K Speed ~108 tok/s
MoE — speed based on ~1.86 GB active experts plus per-token routing overhead (full 10.83 GB loaded in VRAM)
Q8_0 Context 31K Speed ~106 tok/s
MoE — speed based on ~1.93 GB active experts plus per-token routing overhead (full 11.28 GB loaded in VRAM)
Q4_K_M Context 128K Speed ~78 tok/s
Q8_0 Context 84K Speed ~49 tok/s
Q4_K_M Context 128K Speed ~78 tok/s
Q8_0 Context 84K Speed ~49 tok/s
Q4_K_M Context 85K Speed ~53 tok/s
Q8_0 Tight fit · 5K context
Q4_K_M Context 19K Speed ~44 tok/s
Q8_0 Doesn't fit · needs 14.8 GB
Q4_K_M Context 49K Speed ~60 tok/s
Q8_0 Tight fit · 13K context
Q4_K_M Context 19K Speed ~44 tok/s
Q8_0 Doesn't fit · needs 14.8 GB
Q4_K_M Context 26K Speed ~131 tok/s
Q8_0 Context 22K Speed ~88 tok/s
Q4_K_M Context 79K Speed ~127 tok/s
Q8_0 Context 67K Speed ~87 tok/s
Q4_K_M Context 128K Speed ~69 tok/s
Q8_0 Context 128K Speed ~49 tok/s
Q4_K_M Context 60K Speed ~75 tok/s
Q8_0 Context 33K Speed ~47 tok/s
fp16 Doesn't fit · needs 15.1 GB
Q4_K_M Context 128K Speed ~78 tok/s
Q8_0 Context 84K Speed ~49 tok/s
Q4_K_M Context 26K Speed ~52 tok/s
Q8_0 Doesn't fit · needs 12.3 GB
Q4_K_M Context 128K Speed ~151 tok/s
Q8_0 Context 128K Speed ~104 tok/s
Q4_K_M Context 32K Speed ~151 tok/s
Q8_0 Context 32K Speed ~104 tok/s
Q4_K_M Context 50K Speed ~69 tok/s
Q8_0 Context 29K Speed ~47 tok/s
Q4_K_M Context 32K Speed ~143 tok/s
Q8_0 Context 32K Speed ~98 tok/s
Q4_K_M Context 94K Speed ~147 tok/s
Q8_0 Context 82K Speed ~100 tok/s
fp16 Context 56K Speed ~60 tok/s
Q4_K_M Context 128K Speed ~127 tok/s
Q8_0 Context 128K Speed ~86 tok/s
Q4_K_M Context 128K Speed ~100 tok/s
Q8_0 Context 128K Speed ~74 tok/s
Q4_K_M Context 128K Speed ~78 tok/s
Q8_0 Context 84K Speed ~49 tok/s
Q4_K_M Context 256K Speed ~195 tok/s
Q8_0 Context 256K Speed ~148 tok/s
Q4_K_M Context 16K Speed ~41 tok/s
Q8_0 Doesn't fit · needs 15.9 GB
Q4_K_M Context 16K Speed ~87 tok/s
Q8_0 Tight fit · 10K context
Q4_K_M Context 128K Speed ~246 tok/s
Q8_0 Context 128K Speed ~191 tok/s
Q4_K_M Context 256K Speed ~289 tok/s
Q8_0 Context 256K Speed ~244 tok/s
Q4_K_M Context 256K Speed ~112 tok/s
Q8_0 Context 256K Speed ~85 tok/s
Q4_K_M Context 256K Speed ~212 tok/s
Q8_0 Context 256K Speed ~156 tok/s
Q4_K_M Context 213K Speed ~66 tok/s
Q8_0 Context 93K Speed ~41 tok/s
Q4_K_M Context 211K Speed ~65 tok/s
Q8_0 Context 100K Speed ~42 tok/s
Q4_K_M Context 128K Speed ~166 tok/s
Q8_0 Context 128K Speed ~115 tok/s
Q4_K_M Context 128K Speed ~118 tok/s
MoE — speed based on ~1.55 GB active experts plus per-token routing overhead (full 7.52 GB loaded in VRAM)
Q8_0 Doesn't fit · needs 12.1 GB
Q4_K_M Context 128K Speed ~212 tok/s
Q8_0 Context 128K Speed ~156 tok/s
Q4_K_M Context 64K Speed ~62 tok/s
Q8_0 Context 64K Speed ~49 tok/s
Q4_K_M Context 49K Speed ~67 tok/s
Q8_0 Context 22K Speed ~42 tok/s
Tight fit 1 Q4_K_M Context 8K Speed ~66 tok/s
Only 8K context — under the 16K needed to be practical
Q8_0 Context 8K Speed ~41 tok/s
Only 8K context — under the 16K needed to be practical
Notify me when new models fit my GPU An email when a new model fits your 12 GB setup — no spam, unsubscribe anytime.
Doesn't fit 64 Q4_K_M Needs 381.2 GB — 369.2 GB over
Q8_0 Needs 664.4 GB — 652.4 GB over
Q4_K_M Needs 45.1 GB — 33.1 GB over
Q8_0 Needs 79.0 GB — 67.0 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 586.5 GB — 574.5 GB over
Q8_0 Needs 1013.0 GB — 1001.0 GB over
View all 64 →
Doesn't fit 64 ✕ Q4_K_M Needs 381.2 GB — 369.2 GB over
Q8_0 Needs 664.4 GB — 652.4 GB over
Q4_K_M Needs 45.1 GB — 33.1 GB over
Q8_0 Needs 79.0 GB — 67.0 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 586.5 GB — 574.5 GB over
Q8_0 Needs 1013.0 GB — 1001.0 GB over
Q4_K_M Needs 68.2 GB — 56.2 GB over
Q8_0 Needs 118.3 GB — 106.3 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 128.9 GB — 116.9 GB over
Q8_0 Needs 226.5 GB — 214.5 GB over
Q4_K_M Needs 900.1 GB — 888.1 GB over
Q8_0 Needs 1670.1 GB — 1658.1 GB over
Q4_K_M Needs 227.4 GB — 215.4 GB over
Q8_0 Needs 392.6 GB — 380.6 GB over
GLM-4.6 357B Excellent Reasoning Tool use MMLU 87
Q4_K_M Needs 200.8 GB — 188.8 GB over
Q8_0 Needs 353.3 GB — 341.3 GB over
Q4_K_M Needs 428.2 GB — 416.2 GB over
Q8_0 Needs 746.4 GB — 734.4 GB over
GLM 5.3 753.33B Excellent Tool use Reasoning MMLU 86.8
Q8_0 Needs 746.4 GB — 734.4 GB over
Q4_K_M Needs 161.1 GB — 149.1 GB over
Q8_0 Needs 282.1 GB — 270.1 GB over
Q4_K_M Needs 15.5 GB — 3.5 GB over
Q8_0 Needs 28.1 GB — 16.1 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 176.0 GB — 164.0 GB over
Q8_0 Needs 310.4 GB — 298.4 GB over
Q4_K_M Needs 44.5 GB — 32.5 GB over
Q8_0 Needs 72.3 GB — 60.3 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 44.5 GB — 32.5 GB over
Q8_0 Needs 72.3 GB — 60.3 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 15.7 GB — 3.7 GB over
Q8_0 Needs 26.7 GB — 14.7 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 39.9 GB — 27.9 GB over
Q8_0 Needs 70.1 GB — 58.1 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 226.3 GB — 214.3 GB over
Q8_0 Needs 396.8 GB — 384.8 GB over
Q4_K_M Needs 20.5 GB — 8.5 GB over
Q8_0 Needs 34.4 GB — 22.4 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 20.6 GB — 8.6 GB over
Q8_0 Needs 34.4 GB — 22.4 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 17.1 GB — 5.1 GB over
Q8_0 Needs 30.5 GB — 18.5 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 71.3 GB — 59.3 GB over
Q8_0 Needs 121.0 GB — 109.0 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 18.6 GB — 6.6 GB over
Q8_0 Needs 32.7 GB — 20.7 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 15.4 GB — 3.4 GB over
Q8_0 Needs 27.1 GB — 15.1 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 111.9 GB — 99.9 GB over
Q8_0 Needs 195.2 GB — 183.2 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 39.9 GB — 27.9 GB over
Q8_0 Needs 70.1 GB — 58.1 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 18.7 GB — 6.7 GB over
Q8_0 Needs 32.7 GB — 20.7 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
GLM 5 753.86B Excellent Tool use MMLU 83
Q4_K_M Needs 424.7 GB — 412.7 GB over
Q8_0 Needs 746.4 GB — 734.4 GB over
QwQ 32B 32.5B Excellent Reasoning Tool use MMLU 83
Q4_K_M Needs 18.7 GB — 6.7 GB over
Q8_0 Needs 32.7 GB — 20.7 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 175.9 GB — 163.9 GB over
Q8_0 Needs 306.8 GB — 294.8 GB over
Q4_K_M Needs 15.9 GB — 3.9 GB over
Q8_0 Needs 25.0 GB — 13.0 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 129.1 GB — 117.1 GB over
Q8_0 Needs 226.7 GB — 214.7 GB over
Q4_K_M Needs 17.1 GB — 5.1 GB over
Q8_0 Needs 29.7 GB — 17.7 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 13.5 GB — 1.5 GB over
Q8_0 Needs 23.5 GB — 11.5 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 23.7 GB — 11.7 GB over
Q8_0 Needs 32.6 GB — 20.6 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 17.4 GB — 5.4 GB over
Q8_0 Needs 30.3 GB — 18.3 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 13.5 GB — 1.5 GB over
Q8_0 Needs 23.5 GB — 11.5 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 18.7 GB — 6.7 GB over
Q8_0 Needs 32.7 GB — 20.7 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 23.0 GB — 11.0 GB over
Q8_0 Needs 31.3 GB — 19.3 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 61.1 GB — 49.1 GB over
Q8_0 Needs 106.9 GB — 94.9 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 18.7 GB — 6.7 GB over
Q8_0 Needs 32.7 GB — 20.7 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 15.5 GB — 3.5 GB over
Q8_0 Needs 26.8 GB — 14.8 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 15.9 GB — 3.9 GB over
Q8_0 Needs 27.3 GB — 15.3 GB over
Q4_K_M Needs 111.4 GB — 99.4 GB over
Q8_0 Needs 175.4 GB — 163.4 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 379.1 GB — 367.1 GB over
Q8_0 Needs 666.6 GB — 654.6 GB over
Q4_K_M Needs 13.5 GB — 1.5 GB over
Q8_0 Needs 23.5 GB — 11.5 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 12.6 GB — 0.6 GB over
Q8_0 Needs 22.2 GB — 10.2 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 591.3 GB — 579.3 GB over
Q8_0 Needs 1038.6 GB — 1026.6 GB over
Q4_K_M Needs 465.3 GB — 453.3 GB over
Q8_0 Needs 811.9 GB — 799.9 GB over
Q4_K_M Needs 333.4 GB — 321.4 GB over
Q8_0 Needs 566.6 GB — 554.6 GB over
Q4_K_M Needs 293.0 GB — 281.0 GB over
Q8_0 Needs 496.2 GB — 484.2 GB over
Q4_K_M Needs 227.6 GB — 215.6 GB over
Q8_0 Needs 399.1 GB — 387.1 GB over
Q4_K_S Needs 104.0 GB — 92.0 GB over
Q8_0 Needs 195.2 GB — 183.2 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 70.7 GB — 58.7 GB over
Q8_0 Needs 123.5 GB — 111.5 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 65.3 GB — 53.3 GB over
Q8_0 Needs 119.7 GB — 107.7 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q8_0 Needs 73.4 GB — 61.4 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 23.6 GB — 11.6 GB over
Q8_0 Needs 39.5 GB — 27.5 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 20.2 GB — 8.2 GB over
Q8_0 Needs 35.2 GB — 23.2 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 19.7 GB — 7.7 GB over
Q8_0 Needs 34.4 GB — 22.4 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 19.2 GB — 7.2 GB over
Q8_0 Needs 33.2 GB — 21.2 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 19.6 GB — 7.6 GB over
Q8_0 Needs 33.0 GB — 21.0 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 15.6 GB — 3.6 GB over
Q8_0 Needs 27.6 GB — 15.6 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Arete 27B 27.36B Basic Vision Tool use Reasoning MMLU
Q4_K_M Needs 16.1 GB — 4.1 GB over
Q8_0 Needs 26.8 GB — 14.8 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.
Q4_K_M Needs 13.5 GB — 1.5 GB over
Q8_0 Needs 23.8 GB — 11.8 GB over
GPUs that can run this As an Amazon Associate, I earn from qualifying purchases.