What Models?

Pick the right model for your GPU — in seconds.

Best local LLMs for 128 GB VRAM (November 2026): what fits, context and speed

With 128 GB of VRAM or unified memory, 85 models run well and 2 more are a tight fit. 20 don't fit.

Speeds assume 400 GB/s, the median memory bandwidth of the 9 GPUs and memory configurations with exactly 128 GB. Yours may be faster or slower: pick your GPU for exact numbers.

Best overall

Highest MMLU-Pro score among models that run well

  1. Qwen3 Next 80B-A3B Q8_0 256K context · ~77 tok/s · MMLU 90
  2. Mistral Small 4 Q8_0 18K context · ~45 tok/s · MMLU 88.5
  3. Qwen3.6 27B Q8_0 256K context · ~13 tok/s · MMLU 86.2
  4. Qwen 2.5 72B Q8_0 128K context · ~5 tok/s · MMLU 86.1
  5. Qwen 2.5 VL 72B Q8_0 128K context · ~5 tok/s · MMLU 86.1

Best for coding

Highest SWE-bench Verified score among models that run well

  1. Qwen3.8 27B Q8_0 256K context · ~13 tok/s · SWE-bench 86
  2. Ornith 1.5 122B Q8_0 197K context · ~32 tok/s · SWE-bench 84.5
  3. Ornith 1.5 35B A3B Q8_0 256K context · ~78 tok/s · SWE-bench 79
  4. Qwen3.6 27B Q8_0 256K context · ~13 tok/s · SWE-bench 77.2
  5. Step 3.7 Flash Q4_K_S 140K context · ~49 tok/s · SWE-bench 76.5

GPUs with 128 GB of memory

Graphics cards

Unified memory (Apple Silicon, Ryzen AI Max, RTX Spark)

Every model in 128 GB

172 of 214 model configurations can run on 128 GB — sorted by quality

107 models, each counted once by the best way it runs on your hardware.

Largest that runs well Step 3.7 Flash 198B params · Q4_K_S · 140K context
Fastest good-quality model Phi-3 Mini 3.8B ~118 tok/s · Q4_K_M · 128K context
? MMLU (Massive Multitask Language Understanding) measures general knowledge across 57 subjects. Higher = more capable. Sorted best-first.

Runs well 85

  • Qwen3 Next 80B-A3B 80B Excellent Tool use
    MMLU 90
    Q4_K_M
    Context 256K Speed ~105 tok/s

    MoE — speed based on ~1.69 GB active experts plus per-token routing overhead (full 45.09 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~77 tok/s

    MoE — speed based on ~2.96 GB active experts plus per-token routing overhead (full 78.99 GB loaded in VRAM)

  • Mistral Small 4 119B Excellent VisionTool use
    MMLU 88.5
    Q4_K_M
    Context 109K Speed ~67 tok/s

    MoE — speed based on ~3.7 GB active experts plus per-token routing overhead (full 67.65 GB loaded in VRAM)

    Q8_0
    Context 18K Speed ~45 tok/s

    MoE — speed based on ~6.43 GB active experts plus per-token routing overhead (full 117.79 GB loaded in VRAM)

  • Qwen3.6 27B 27B Excellent VisionTool useReasoning
    MMLU 86.2
    Q4_K_M
    Context 256K Speed ~23 tok/s
    Q8_0
    Context 256K Speed ~13 tok/s

Runs well 85

  • Qwen3 Next 80B-A3B 80B Excellent Tool use
    MMLU 90
    Q4_K_M
    Context 256K Speed ~105 tok/s

    MoE — speed based on ~1.69 GB active experts plus per-token routing overhead (full 45.09 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~77 tok/s

    MoE — speed based on ~2.96 GB active experts plus per-token routing overhead (full 78.99 GB loaded in VRAM)

  • Mistral Small 4 119B Excellent VisionTool use
    MMLU 88.5
    Q4_K_M
    Context 109K Speed ~67 tok/s

    MoE — speed based on ~3.7 GB active experts plus per-token routing overhead (full 67.65 GB loaded in VRAM)

    Q8_0
    Context 18K Speed ~45 tok/s

    MoE — speed based on ~6.43 GB active experts plus per-token routing overhead (full 117.79 GB loaded in VRAM)

  • Qwen3.6 27B 27B Excellent VisionTool useReasoning
    MMLU 86.2
    Q4_K_M
    Context 256K Speed ~23 tok/s
    Q8_0
    Context 256K Speed ~13 tok/s
  • Qwen3.5 27B 27B Excellent VisionTool useReasoning
    MMLU 86.1
    Q4_K_M
    Context 256K Speed ~23 tok/s
    Q8_0
    Context 256K Speed ~14 tok/s
  • Qwen 2.5 72B 72.7B Excellent Tool use
    MMLU 86.1
    Q4_K_M
    Context 128K Speed ~8 tok/s
    Q8_0
    Context 128K Speed ~5 tok/s
  • Qwen 2.5 VL 72B 72.7B Excellent VisionTool use
    MMLU 86.1
    Q4_K_M
    Context 128K Speed ~8 tok/s
    Q8_0
    Context 128K Speed ~5 tok/s
  • Llama 3.3 70B 70.6B Excellent Tool use
    MMLU 86
    Q4_K_M
    Context 128K Speed ~9 tok/s
    Q8_0
    Context 128K Speed ~5 tok/s
  • Qwen3.5 35B A3B 35B Excellent VisionTool useReasoning
    MMLU 85.3
    Q4_K_M
    Context 256K Speed ~103 tok/s

    MoE — speed based on ~1.76 GB active experts plus per-token routing overhead (full 20.5 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~77 tok/s

    MoE — speed based on ~2.95 GB active experts plus per-token routing overhead (full 34.37 GB loaded in VRAM)

  • Gemma 4 31B 31B Excellent VisionTool useReasoning
    MMLU 85.2
    Q4_K_M
    Context 256K Speed ~21 tok/s
    Q8_0
    Context 256K Speed ~12 tok/s
  • Qwen3.6 35B A3B 35B Excellent VisionTool useReasoning
    MMLU 85.2
    Q4_K_M
    Context 256K Speed ~103 tok/s

    MoE — speed based on ~1.77 GB active experts plus per-token routing overhead (full 20.61 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~77 tok/s

    MoE — speed based on ~2.95 GB active experts plus per-token routing overhead (full 34.37 GB loaded in VRAM)

  • Qwen3.5 122B A10B 122B Excellent VisionTool useReasoning
    MMLU 85
    Q4_K_M
    Context 256K Speed ~48 tok/s

    MoE — speed based on ~5.84 GB active experts plus per-token routing overhead (full 71.28 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~32 tok/s

    MoE — speed based on ~9.91 GB active experts plus per-token routing overhead (full 120.95 GB loaded in VRAM)

  • Qwen3 32B 32.8B Excellent ReasoningTool use
    MMLU 85
    Q4_K_M
    Context 128K Speed ~19 tok/s
    Q8_0
    Context 128K Speed ~11 tok/s
  • Phi-4 14B 14.7B Excellent
    MMLU 84.8
    Q4_K_M
    Context 16K Speed ~40 tok/s
    Q8_0
    Context 16K Speed ~24 tok/s
  • Qwen3.8 27B 27.78B Excellent VisionTool useReasoning
    MMLU 84.3
    Q4_K_M
    Context 256K Speed ~23 tok/s
    Q8_0
    Context 256K Speed ~13 tok/s
  • DeepSeek R1 Distill Llama 70B 70.6B Excellent Reasoning
    MMLU 84
    Q4_K_M
    Context 128K Speed ~9 tok/s
    Q8_0
    Context 128K Speed ~5 tok/s
  • Step 3.5 Flash 197B Excellent ReasoningTool use
    MMLU 84
    Q4_K_M
    Context 94K Speed ~46 tok/s

    MoE — speed based on ~6.24 GB active experts plus per-token routing overhead (full 111.7 GB loaded in VRAM)

    Q8_0 Doesn't fit · needs 195.2 GB
  • Qwen 2.5 32B 32.5B Excellent Tool use
    MMLU 83.3
    Q4_K_M
    Context 128K Speed ~19 tok/s
    Q8_0
    Context 128K Speed ~11 tok/s
  • QwQ 32B 32.5B Excellent ReasoningTool use
    MMLU 83
    Q4_K_M
    Context 128K Speed ~19 tok/s
    Q8_0
    Context 128K Speed ~11 tok/s
  • Gemma 4 26B A4B 26B Great VisionTool useReasoning
    MMLU 82.6
    Q4_K_M
    Context 256K Speed ~87 tok/s

    MoE — speed based on ~2.44 GB active experts plus per-token routing overhead (full 15.87 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~65 tok/s

    MoE — speed based on ~3.85 GB active experts plus per-token routing overhead (full 25.02 GB loaded in VRAM)

  • Qwen3.5 9B 9B Great VisionTool useReasoning
    MMLU 82.5
    Q4_K_M
    Context 256K Speed ~60 tok/s
    Q8_0
    Context 256K Speed ~38 tok/s
  • GLM-4.7 Flash 31B Great ReasoningTool use
    MMLU 82
    Q4_K_M
    Context 128K Speed ~106 tok/s

    MoE — speed based on ~1.65 GB active experts plus per-token routing overhead (full 17.05 GB loaded in VRAM)

    Q8_0
    Context 128K Speed ~79 tok/s

    MoE — speed based on ~2.87 GB active experts plus per-token routing overhead (full 29.66 GB loaded in VRAM)

  • Magistral Small 24B 23.6B Great ReasoningVisionTool use
    MMLU 82
    Q4_K_M
    Context 128K Speed ~26 tok/s
    Q8_0
    Context 128K Speed ~15 tok/s
  • Nemotron 3.5 Lightning 30B A3B 31.58B Great Tool useReasoning
    MMLU 81.94
    Q4_K_M
    Context 256K Speed ~91 tok/s

    MoE — speed based on ~2.25 GB active experts plus per-token routing overhead (full 23.73 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~75 tok/s

    MoE — speed based on ~3.1 GB active experts plus per-token routing overhead (full 32.6 GB loaded in VRAM)

  • Qwen3 Coder 30B-A3B 30.5B Great Tool use
    MMLU 81
    Q4_K_M
    Context 256K Speed ~105 tok/s

    MoE — speed based on ~1.7 GB active experts plus per-token routing overhead (full 17.28 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~77 tok/s

    MoE — speed based on ~2.98 GB active experts plus per-token routing overhead (full 30.25 GB loaded in VRAM)

  • Mistral Small 3.1 24B 23.6B Great VisionTool use
    MMLU 81
    Q4_K_M
    Context 128K Speed ~26 tok/s
    Q8_0
    Context 128K Speed ~15 tok/s
  • DeepSeek R1 Distill Qwen 32B 32.5B Great Reasoning
    MMLU 80
    Q4_K_M
    Context 128K Speed ~19 tok/s
    Q8_0
    Context 128K Speed ~11 tok/s
  • Qwen 2.5 14B 14.7B Great Tool use
    MMLU 79.9
    Q4_K_M
    Context 128K Speed ~40 tok/s
    Q8_0
    Context 128K Speed ~24 tok/s
  • Nemotron Cascade 2 30B A3B 30B Great ReasoningTool use
    MMLU 79.8
    Q4_K_M
    Context 256K Speed ~89 tok/s

    MoE — speed based on ~2.3 GB active experts plus per-token routing overhead (full 23.03 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~75 tok/s

    MoE — speed based on ~3.13 GB active experts plus per-token routing overhead (full 31.28 GB loaded in VRAM)

  • Llama 4 Scout 109B Great VisionTool use
    MMLU 79.6
    Q4_K_M
    Context 128K Speed ~33 tok/s

    MoE — speed based on ~9.49 GB active experts plus per-token routing overhead (full 60.87 GB loaded in VRAM)

    Q8_0
    Context 116K Speed ~20 tok/s

    MoE — speed based on ~16.64 GB active experts plus per-token routing overhead (full 106.67 GB loaded in VRAM)

  • Qwen3.5 4B 4B Great VisionTool useReasoning
    MMLU 79.1
    Q4_K_M
    Context 256K Speed ~107 tok/s
    Q8_0
    Context 256K Speed ~73 tok/s
  • Qwen3 8B 8.2B Great ReasoningTool use
    MMLU 79
    Q4_K_M
    Context 128K Speed ~66 tok/s
    Q8_0
    Context 128K Speed ~41 tok/s
  • Qwen 2.5 Coder 32B 32.5B Great
    MMLU 79
    Q4_K_M
    Context 128K Speed ~19 tok/s
    Q8_0
    Context 128K Speed ~11 tok/s
  • Gemma 3 27B 27B Great Vision
    MMLU 78
    Q4_K_M
    Context 128K Speed ~23 tok/s
    Q8_0
    Context 128K Speed ~13 tok/s
  • Gemma 4 12B 11.96B Great VisionTool useReasoning
    MMLU 77.2
    Q4_K_M
    Context 256K Speed ~49 tok/s
    Q8_0
    Context 256K Speed ~29 tok/s
  • Nemotron Nano 12B 12.6B Great ReasoningTool use
    MMLU 76
    Q4_K_M
    Context 128K Speed ~47 tok/s
    Q8_0
    Context 128K Speed ~28 tok/s
  • GPT-OSS 20B 21B Great ReasoningTool use
    MMLU 75
    Q4_K_M
    Context 128K Speed ~100 tok/s

    MoE — speed based on ~1.86 GB active experts plus per-token routing overhead (full 10.83 GB loaded in VRAM)

    Q8_0
    Context 128K Speed ~98 tok/s

    MoE — speed based on ~1.93 GB active experts plus per-token routing overhead (full 11.28 GB loaded in VRAM)

  • Qwen 2.5 7B 7.62B Good Tool use
    MMLU 74.2
    Q4_K_M
    Context 128K Speed ~70 tok/s
    Q8_0
    Context 128K Speed ~44 tok/s
  • Qwen 2.5 VL 7B 7.62B Good VisionTool use
    MMLU 74.2
    Q4_K_M
    Context 128K Speed ~70 tok/s
    Q8_0
    Context 128K Speed ~44 tok/s
  • Gemma 3 12B 11.8B Good Vision
    MMLU 74
    Q4_K_M
    Context 128K Speed ~48 tok/s
    Q8_0
    Context 128K Speed ~30 tok/s
  • Qwen 2.5 Coder 14B 14.7B Good
    MMLU 74
    Q4_K_M
    Context 128K Speed ~40 tok/s
    Q8_0
    Context 128K Speed ~24 tok/s
  • Qwen3.8 Flash Next 180B Good VisionTool useReasoning
    MMLU 73.23
    Q4_K_M
    Context 256K Speed ~67 tok/s

    MoE — speed based on ~3.71 GB active experts plus per-token routing overhead (full 111.39 GB loaded in VRAM)

    Q8_0 Doesn't fit · needs 175.4 GB
  • Llama 3.2 Vision 11B 10.6B Good VisionTool use
    MMLU 73
    Q4_K_M
    Context 128K Speed ~54 tok/s
    Q8_0
    Context 128K Speed ~33 tok/s
  • DeepSeek R1 Distill Qwen 14B 14.7B Good Reasoning
    MMLU 72
    Q4_K_M
    Context 128K Speed ~40 tok/s
    Q8_0
    Context 128K Speed ~24 tok/s
  • Devstral Small 24B 23.6B Good Tool use
    MMLU 72
    Q4_K_M
    Context 128K Speed ~26 tok/s
    Q8_0
    Context 128K Speed ~15 tok/s
  • Phi-3 Mini 3.8B 3.82B Good
    MMLU 70.9
    Q4_K_M
    Context 128K Speed ~118 tok/s
    Q8_0
    Context 128K Speed ~79 tok/s
  • Phi-4 Mini 3.8B 3.84B Good
    MMLU 70
    Q4_K_M
    Context 128K Speed ~115 tok/s
    Q8_0
    Context 128K Speed ~79 tok/s
  • Llama 3.1 8B 8.03B Good Tool use
    MMLU 69.4
    Q4_K_M
    Context 128K Speed ~68 tok/s
    Q8_0
    Context 128K Speed ~42 tok/s
    fp16
    Context 128K Speed ~23 tok/s
  • Gemma 4 E4B 4.5B Good VisionTool useReasoning
    MMLU 69.4
    Q4_K_M
    Context 128K Speed ~62 tok/s
    Q8_0
    Context 128K Speed ~44 tok/s
  • Mistral Nemo 12B 12.2B Good Tool use
    MMLU 68
    Q4_K_M
    Context 128K Speed ~47 tok/s
    Q8_0
    Context 128K Speed ~29 tok/s
  • Qwen 2.5 Coder 7B 7.62B Good
    MMLU 68
    Q4_K_M
    Context 128K Speed ~70 tok/s
    Q8_0
    Context 128K Speed ~44 tok/s
  • Qwen 2.5 VL 3B 3.09B Fair Vision
    MMLU 65.6
    Q4_K_M
    Context 128K Speed ~136 tok/s
    Q8_0
    Context 128K Speed ~93 tok/s
  • Qwen 2.5 3B 3.09B Fair Tool use
    MMLU 65.6
    Q4_K_M
    Context 32K Speed ~136 tok/s
    Q8_0
    Context 32K Speed ~93 tok/s
  • Ministral 8B 8.02B Fair Tool use
    MMLU 65
    Q4_K_M
    Context 128K Speed ~62 tok/s
    Q8_0
    Context 128K Speed ~42 tok/s
  • SmallThinker 3B 3.09B Fair Reasoning
    MMLU 65
    Q4_K_M
    Context 32K Speed ~129 tok/s
    Q8_0
    Context 32K Speed ~88 tok/s
  • Llama 3.2 3B 3.21B Fair Tool use
    MMLU 63.4
    Q4_K_M
    Context 128K Speed ~133 tok/s
    Q8_0
    Context 128K Speed ~90 tok/s
    fp16
    Context 128K Speed ~54 tok/s
  • Gemma 3 4B 3.88B Fair Vision
    MMLU 63
    Q4_K_M
    Context 128K Speed ~115 tok/s
    Q8_0
    Context 128K Speed ~78 tok/s
  • Codestral 22B 22.2B Fair
    MMLU 62
    Q4_K_M
    Context 32K Speed ~28 tok/s
    Q8_0
    Context 32K Speed ~16 tok/s
  • Gemma 4 E2B 2.3B Fair VisionTool useReasoning
    MMLU 60
    Q4_K_M
    Context 128K Speed ~90 tok/s
    Q8_0
    Context 128K Speed ~67 tok/s
  • DeepSeek R1 Distill Qwen 7B 7.62B Fair Reasoning
    MMLU 58
    Q4_K_M
    Context 128K Speed ~70 tok/s
    Q8_0
    Context 128K Speed ~44 tok/s
  • Qwen3.5 2B 2B Fair VisionTool useReasoning
    MMLU 55.3
    Q4_K_M
    Context 256K Speed ~176 tok/s
    Q8_0
    Context 256K Speed ~133 tok/s
  • StarCoder2 15B 15.6B Basic
    MMLU 51.4
    Q4_K_M
    Context 16K Speed ~37 tok/s
    Q8_0
    Context 16K Speed ~22 tok/s
  • DeepSeek Coder 6.7B 6.74B Basic
    MMLU 50
    Q4_K_M
    Context 16K Speed ~79 tok/s
    Q8_0
    Context 16K Speed ~49 tok/s
  • Llama 3.2 1B 1.24B Basic Tool use
    MMLU 49.3
    Q4_K_M
    Context 128K Speed ~222 tok/s
    Q8_0
    Context 128K Speed ~172 tok/s
  • Qwen3.5 0.8B 0.8B Basic VisionTool useReasoning
    MMLU 29.7
    Q4_K_M
    Context 256K Speed ~260 tok/s
    Q8_0
    Context 256K Speed ~220 tok/s
  • Nemotron 3.5 Super VL 120B A12B 124.45B Basic VisionTool useReasoning
    MMLU
    Q4_K_M
    Context 1M Speed ~46 tok/s

    MoE — speed based on ~6.3 GB active experts plus per-token routing overhead (full 65.29 GB loaded in VRAM)

    Q8_0
    Context 1M Speed ~28 tok/s

    MoE — speed based on ~11.54 GB active experts plus per-token routing overhead (full 119.65 GB loaded in VRAM)

  • K2-Horizon 7B 9B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 512K Speed ~61 tok/s
    Q8_0
    Context 512K Speed ~38 tok/s
  • Nanbeige4.3 33B 33.12B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 512K Speed ~18 tok/s
    Q8_0
    Context 480K Speed ~11 tok/s
  • Nemotron 3 Nano 4B 3.97B Basic ReasoningTool use
    MMLU
    Q4_K_M
    Context 256K Speed ~101 tok/s
    Q8_0
    Context 256K Speed ~76 tok/s
  • Ornith 1.5 9B 9.65B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~59 tok/s
    Q8_0
    Context 256K Speed ~37 tok/s
  • Ornith 1.5 35B A3B 35.95B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~105 tok/s

    MoE — speed based on ~1.69 GB active experts plus per-token routing overhead (full 20.22 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~78 tok/s

    MoE — speed based on ~2.94 GB active experts plus per-token routing overhead (full 35.21 GB loaded in VRAM)

  • Laguna XS 2.1 33.44B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~104 tok/s

    MoE — speed based on ~1.72 GB active experts plus per-token routing overhead (full 19.14 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~77 tok/s

    MoE — speed based on ~2.97 GB active experts plus per-token routing overhead (full 33.15 GB loaded in VRAM)

  • Nemotron 3.5 Bolt 74B A9B 74.89B Basic VisionTool useReasoning
    MMLU
    Q8_0
    Context 256K Speed ~35 tok/s

    MoE — speed based on ~8.83 GB active experts plus per-token routing overhead (full 73.44 GB loaded in VRAM)

  • Ornith 1.5 122B 125.09B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~50 tok/s

    MoE — speed based on ~5.65 GB active experts plus per-token routing overhead (full 70.64 GB loaded in VRAM)

    Q8_0
    Context 197K Speed ~32 tok/s

    MoE — speed based on ~9.87 GB active experts plus per-token routing overhead (full 123.45 GB loaded in VRAM)

  • Flint 40B A2.5B 39.79B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~111 tok/s

    MoE — speed based on ~1.48 GB active experts plus per-token routing overhead (full 23.61 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~86 tok/s

    MoE — speed based on ~2.48 GB active experts plus per-token routing overhead (full 39.44 GB loaded in VRAM)

  • MiniCPM-V 4.7 35B A3B 35.21B Basic VisionTool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~105 tok/s

    MoE — speed based on ~1.68 GB active experts plus per-token routing overhead (full 19.71 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~78 tok/s

    MoE — speed based on ~2.93 GB active experts plus per-token routing overhead (full 34.37 GB loaded in VRAM)

  • MiMo V2.6 Distill Qwen 9B 9.41B Basic VisionTool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~58 tok/s
    Q8_0
    Context 256K Speed ~38 tok/s
  • LFM3 1.7B 1.73B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~191 tok/s
    Q8_0
    Context 256K Speed ~141 tok/s
  • Arete 27B 27.36B Basic VisionTool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~22 tok/s
    Q8_0
    Context 256K Speed ~13 tok/s
  • Step 3.7 Flash 198B Basic VisionTool useReasoning
    MMLU
    Q4_K_S
    Context 140K Speed ~49 tok/s

    MoE — speed based on ~5.77 GB active experts plus per-token routing overhead (full 103.84 GB loaded in VRAM)

    Q8_0 Doesn't fit · needs 195.2 GB
  • Muse Glimmer 30B 29.78B Basic VisionTool useReasoning
    MMLU
    Q4_K_M
    Context 128K Speed ~23 tok/s
    Q8_0
    Context 128K Speed ~13 tok/s
  • LFM2.5 2.6B 2.7B Basic Tool use
    MMLU
    Q4_K_M
    Context 128K Speed ~150 tok/s
    Q8_0
    Context 128K Speed ~103 tok/s
  • LFM2.5 24B A2B 23.97B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 128K Speed ~115 tok/s

    MoE — speed based on ~1.38 GB active experts plus per-token routing overhead (full 13.53 GB loaded in VRAM)

    Q8_0
    Context 128K Speed ~87 tok/s

    MoE — speed based on ~2.43 GB active experts plus per-token routing overhead (full 23.74 GB loaded in VRAM)

  • Mellum2.1 12B A2.5B Thinking 12.15B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 128K Speed ~109 tok/s

    MoE — speed based on ~1.55 GB active experts plus per-token routing overhead (full 7.52 GB loaded in VRAM)

    Q8_0
    Context 128K Speed ~86 tok/s

    MoE — speed based on ~2.48 GB active experts plus per-token routing overhead (full 12.04 GB loaded in VRAM)

  • LFM3-VL 2B 1.92B Basic VisionTool useReasoning
    MMLU
    Q4_K_M
    Context 128K Speed ~191 tok/s
    Q8_0
    Context 128K Speed ~141 tok/s
  • Nemotron 3.5 Bolt E4B 8B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 64K Speed ~56 tok/s
    Q8_0
    Context 64K Speed ~44 tok/s

Tight fit 2

  • Gemma 2 27B 27.2B Great
    MMLU 75.2
    Q4_K_M
    Context 8K Speed ~23 tok/s

    Only 8K context — under the 16K needed to be practical

    Q8_0
    Context 8K Speed ~13 tok/s

    Only 8K context — under the 16K needed to be practical

  • Gemma 2 9B 9.24B Good
    MMLU 71.3
    Q4_K_M
    Context 8K Speed ~59 tok/s

    Only 8K context — under the 16K needed to be practical

    Q8_0
    Context 8K Speed ~37 tok/s

    Only 8K context — under the 16K needed to be practical

Notify me when new models fit my GPU

An email when a new model fits your 128 GB setup — no spam, unsubscribe anytime.

Doesn't fit 20

  • DeepSeek R1 0528 671B Excellent Reasoning
    MMLU 90.8
    Q4_K_M Needs 381.2 GB — 253.2 GB over
    Q8_0 Needs 664.4 GB — 536.4 GB over
  • MiMo V2.5 Pro 1020B Excellent Reasoning
    MMLU 89.4
    Q4_K_M Needs 586.5 GB — 458.5 GB over
    Q8_0 Needs 1013.0 GB — 885.0 GB over
  • MiniMax M2.1 229B Excellent ReasoningTool use
    MMLU 88
    Q4_K_M Needs 128.9 GB — 0.9 GB over
    Q8_0 Needs 226.5 GB — 98.5 GB over

Doesn't fit 20

  • DeepSeek R1 0528 671B Excellent Reasoning
    MMLU 90.8
    Q4_K_M Needs 381.2 GB — 253.2 GB over
    Q8_0 Needs 664.4 GB — 536.4 GB over
  • MiMo V2.5 Pro 1020B Excellent Reasoning
    MMLU 89.4
    Q4_K_M Needs 586.5 GB — 458.5 GB over
    Q8_0 Needs 1013.0 GB — 885.0 GB over
  • MiniMax M2.1 229B Excellent ReasoningTool use
    MMLU 88
    Q4_K_M Needs 128.9 GB — 0.9 GB over
    Q8_0 Needs 226.5 GB — 98.5 GB over
  • DeepSeek V4 Pro 1600B Excellent Tool useReasoning
    MMLU 87.5
    Q4_K_M Needs 900.1 GB — 772.1 GB over
    Q8_0 Needs 1670.1 GB — 1542.1 GB over
  • Qwen3.5 397B A17B 397B Excellent VisionTool useReasoning
    MMLU 87
    Q4_K_M Needs 227.4 GB — 99.4 GB over
    Q8_0 Needs 392.6 GB — 264.6 GB over
  • GLM-4.6 357B Excellent ReasoningTool use
    MMLU 87
    Q4_K_M Needs 200.8 GB — 72.8 GB over
    Q8_0 Needs 353.3 GB — 225.3 GB over
  • GLM 5.1 754B Excellent Tool use
    MMLU 86.9
    Q4_K_M Needs 428.2 GB — 300.2 GB over
    Q8_0 Needs 746.4 GB — 618.4 GB over
  • GLM 5.3 753.33B Excellent Tool useReasoning
    MMLU 86.8
    Q8_0 Needs 746.4 GB — 618.4 GB over
  • DeepSeek V4 Flash 284B Excellent Tool useReasoning
    MMLU 86.4
    Q4_K_M Needs 161.1 GB — 33.1 GB over
    Q8_0 Needs 282.1 GB — 154.1 GB over
  • GLM 5.3 Flash 321.32B Excellent Tool useReasoning
    MMLU 86.1
    Q4_K_M Needs 176.0 GB — 48.0 GB over
    Q8_0 Needs 310.4 GB — 182.4 GB over
  • Llama 4 Maverick 400B Excellent VisionTool use
    MMLU 85.5
    Q4_K_M Needs 226.3 GB — 98.3 GB over
    Q8_0 Needs 396.8 GB — 268.8 GB over
  • GLM 5 753.86B Excellent Tool use
    MMLU 83
    Q4_K_M Needs 424.7 GB — 296.7 GB over
    Q8_0 Needs 746.4 GB — 618.4 GB over
  • MiMo V2.5 310B Great Reasoning
    MMLU 82.9
    Q4_K_M Needs 175.9 GB — 47.9 GB over
    Q8_0 Needs 306.8 GB — 178.8 GB over
  • MiniMax M2.5 228.7B Great ReasoningTool use
    MMLU 82
    Q4_K_M Needs 129.1 GB — 1.1 GB over
    Q8_0 Needs 226.7 GB — 98.7 GB over
  • Mistral Large 3 675B 673.42B Good VisionTool useReasoning
    MMLU 73.11
    Q4_K_M Needs 379.1 GB — 251.1 GB over
    Q8_0 Needs 666.6 GB — 538.6 GB over
  • Mistral Large 4 1T A52B 1049.43B Basic VisionTool useReasoning
    MMLU
    Q4_K_M Needs 591.3 GB — 463.3 GB over
    Q8_0 Needs 1038.6 GB — 910.6 GB over
  • Muse Spark Open 816B A42B 822.04B Basic VisionTool useReasoning
    MMLU
    Q4_K_M Needs 465.3 GB — 337.3 GB over
    Q8_0 Needs 811.9 GB — 683.9 GB over
  • Ling 3.1 Flash 572.12B Basic Tool useReasoning
    MMLU
    Q4_K_M Needs 333.4 GB — 205.4 GB over
    Q8_0 Needs 566.6 GB — 438.6 GB over
  • Beam 1.0 501B A23B 501.27B Basic Tool useReasoning
    MMLU
    Q4_K_M Needs 293.0 GB — 165.0 GB over
    Q8_0 Needs 496.2 GB — 368.2 GB over
  • Ornith 1.5 397B 403.4B Basic Tool useReasoning
    MMLU
    Q4_K_M Needs 227.6 GB — 99.6 GB over
    Q8_0 Needs 399.1 GB — 271.1 GB over