What Models?

Pick the right model for your GPU — in seconds.

Best local LLMs for 96 GB VRAM (November 2026): what fits, context and speed

With 96 GB of VRAM or unified memory, 82 models run well and 2 more are a tight fit. 23 don't fit.

Speeds assume 487 GB/s, the median memory bandwidth of the 6 GPUs and memory configurations with exactly 96 GB. Yours may be faster or slower: pick your GPU for exact numbers.

Best overall

Highest MMLU-Pro score among models that run well

  1. Qwen3 Next 80B-A3B Q8_0 256K context · ~90 tok/s · MMLU 90
  2. Mistral Small 4 Q4_K_M 51K context · ~79 tok/s · MMLU 88.5
  3. Qwen3.6 27B Q8_0 256K context · ~16 tok/s · MMLU 86.2
  4. Qwen 2.5 72B Q8_0 78K context · ~6 tok/s · MMLU 86.1
  5. Qwen 2.5 VL 72B Q8_0 78K context · ~6 tok/s · MMLU 86.1

Best for coding

Highest SWE-bench Verified score among models that run well

  1. Qwen3.8 27B Q8_0 256K context · ~16 tok/s · SWE-bench 86
  2. Ornith 1.5 122B Q4_K_M 256K context · ~59 tok/s · SWE-bench 84.5
  3. Ornith 1.5 35B A3B Q8_0 256K context · ~91 tok/s · SWE-bench 79
  4. Qwen3.6 27B Q8_0 256K context · ~16 tok/s · SWE-bench 77.2
  5. Muse Glimmer 30B Q8_0 128K context · ~16 tok/s · SWE-bench 76

GPUs with 96 GB of memory

Graphics cards

Unified memory (Apple Silicon, Ryzen AI Max, RTX Spark)

Upgrade options

128 GB runs 3 more models well, including Step 3.5 Flash and Qwen3.8 Flash Next.

The cheapest GPUs with more than 96 GB, by launch MSRP.

As an Amazon Associate, I earn from qualifying purchases.

Every model in 96 GB

164 of 214 model configurations can run on 96 GB — sorted by quality

107 models, each counted once by the best way it runs on your hardware.

Largest that runs well Ornith 1.5 122B 125.09B params · Q4_K_M · 256K context
Fastest good-quality model Phi-3 Mini 3.8B ~143 tok/s · Q4_K_M · 128K context
? MMLU (Massive Multitask Language Understanding) measures general knowledge across 57 subjects. Higher = more capable. Sorted best-first.

Runs well 82

  • Qwen3 Next 80B-A3B 80B Excellent Tool use
    MMLU 90
    Q4_K_M
    Context 256K Speed ~121 tok/s

    MoE — speed based on ~1.69 GB active experts plus per-token routing overhead (full 45.09 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~90 tok/s

    MoE — speed based on ~2.96 GB active experts plus per-token routing overhead (full 78.99 GB loaded in VRAM)

  • Mistral Small 4 119B Excellent VisionTool use
    MMLU 88.5
    Q4_K_M
    Context 51K Speed ~79 tok/s

    MoE — speed based on ~3.7 GB active experts plus per-token routing overhead (full 67.65 GB loaded in VRAM)

    Q8_0 Doesn't fit · needs 118.3 GB
  • Qwen3.6 27B 27B Excellent VisionTool useReasoning
    MMLU 86.2
    Q4_K_M
    Context 256K Speed ~28 tok/s
    Q8_0
    Context 256K Speed ~16 tok/s

Runs well 82

  • Qwen3 Next 80B-A3B 80B Excellent Tool use
    MMLU 90
    Q4_K_M
    Context 256K Speed ~121 tok/s

    MoE — speed based on ~1.69 GB active experts plus per-token routing overhead (full 45.09 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~90 tok/s

    MoE — speed based on ~2.96 GB active experts plus per-token routing overhead (full 78.99 GB loaded in VRAM)

  • Mistral Small 4 119B Excellent VisionTool use
    MMLU 88.5
    Q4_K_M
    Context 51K Speed ~79 tok/s

    MoE — speed based on ~3.7 GB active experts plus per-token routing overhead (full 67.65 GB loaded in VRAM)

    Q8_0 Doesn't fit · needs 118.3 GB
  • Qwen3.6 27B 27B Excellent VisionTool useReasoning
    MMLU 86.2
    Q4_K_M
    Context 256K Speed ~28 tok/s
    Q8_0
    Context 256K Speed ~16 tok/s
  • Qwen3.5 27B 27B Excellent VisionTool useReasoning
    MMLU 86.1
    Q4_K_M
    Context 256K Speed ~27 tok/s
    Q8_0
    Context 256K Speed ~16 tok/s
  • Qwen 2.5 72B 72.7B Excellent Tool use
    MMLU 86.1
    Q4_K_M
    Context 128K Speed ~10 tok/s
    Q8_0
    Context 78K Speed ~6 tok/s
  • Qwen 2.5 VL 72B 72.7B Excellent VisionTool use
    MMLU 86.1
    Q4_K_M
    Context 128K Speed ~10 tok/s
    Q8_0
    Context 78K Speed ~6 tok/s
  • Llama 3.3 70B 70.6B Excellent Tool use
    MMLU 86
    Q4_K_M
    Context 128K Speed ~11 tok/s
    Q8_0
    Context 85K Speed ~6 tok/s
  • Qwen3.5 35B A3B 35B Excellent VisionTool useReasoning
    MMLU 85.3
    Q4_K_M
    Context 256K Speed ~119 tok/s

    MoE — speed based on ~1.76 GB active experts plus per-token routing overhead (full 20.5 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~90 tok/s

    MoE — speed based on ~2.95 GB active experts plus per-token routing overhead (full 34.37 GB loaded in VRAM)

  • Gemma 4 31B 31B Excellent VisionTool useReasoning
    MMLU 85.2
    Q4_K_M
    Context 256K Speed ~25 tok/s
    Q8_0
    Context 256K Speed ~14 tok/s
  • Qwen3.6 35B A3B 35B Excellent VisionTool useReasoning
    MMLU 85.2
    Q4_K_M
    Context 256K Speed ~118 tok/s

    MoE — speed based on ~1.77 GB active experts plus per-token routing overhead (full 20.61 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~90 tok/s

    MoE — speed based on ~2.95 GB active experts plus per-token routing overhead (full 34.37 GB loaded in VRAM)

  • Qwen3.5 122B A10B 122B Excellent VisionTool useReasoning
    MMLU 85
    Q4_K_M
    Context 256K Speed ~57 tok/s

    MoE — speed based on ~5.84 GB active experts plus per-token routing overhead (full 71.28 GB loaded in VRAM)

    Q8_0 Doesn't fit · needs 121.0 GB
  • Qwen3 32B 32.8B Excellent ReasoningTool use
    MMLU 85
    Q4_K_M
    Context 128K Speed ~23 tok/s
    Q8_0
    Context 128K Speed ~14 tok/s
  • Phi-4 14B 14.7B Excellent
    MMLU 84.8
    Q4_K_M
    Context 16K Speed ~48 tok/s
    Q8_0
    Context 16K Speed ~29 tok/s
  • Qwen3.8 27B 27.78B Excellent VisionTool useReasoning
    MMLU 84.3
    Q4_K_M
    Context 256K Speed ~28 tok/s
    Q8_0
    Context 256K Speed ~16 tok/s
  • DeepSeek R1 Distill Llama 70B 70.6B Excellent Reasoning
    MMLU 84
    Q4_K_M
    Context 128K Speed ~11 tok/s
    Q8_0
    Context 85K Speed ~6 tok/s
  • Qwen 2.5 32B 32.5B Excellent Tool use
    MMLU 83.3
    Q4_K_M
    Context 128K Speed ~23 tok/s
    Q8_0
    Context 128K Speed ~14 tok/s
  • QwQ 32B 32.5B Excellent ReasoningTool use
    MMLU 83
    Q4_K_M
    Context 128K Speed ~23 tok/s
    Q8_0
    Context 128K Speed ~14 tok/s
  • Gemma 4 26B A4B 26B Great VisionTool useReasoning
    MMLU 82.6
    Q4_K_M
    Context 256K Speed ~101 tok/s

    MoE — speed based on ~2.44 GB active experts plus per-token routing overhead (full 15.87 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~77 tok/s

    MoE — speed based on ~3.85 GB active experts plus per-token routing overhead (full 25.02 GB loaded in VRAM)

  • Qwen3.5 9B 9B Great VisionTool useReasoning
    MMLU 82.5
    Q4_K_M
    Context 256K Speed ~73 tok/s
    Q8_0
    Context 256K Speed ~46 tok/s
  • GLM-4.7 Flash 31B Great ReasoningTool use
    MMLU 82
    Q4_K_M
    Context 128K Speed ~122 tok/s

    MoE — speed based on ~1.65 GB active experts plus per-token routing overhead (full 17.05 GB loaded in VRAM)

    Q8_0
    Context 128K Speed ~92 tok/s

    MoE — speed based on ~2.87 GB active experts plus per-token routing overhead (full 29.66 GB loaded in VRAM)

  • Magistral Small 24B 23.6B Great ReasoningVisionTool use
    MMLU 82
    Q4_K_M
    Context 128K Speed ~32 tok/s
    Q8_0
    Context 128K Speed ~19 tok/s
  • Nemotron 3.5 Lightning 30B A3B 31.58B Great Tool useReasoning
    MMLU 81.94
    Q4_K_M
    Context 256K Speed ~105 tok/s

    MoE — speed based on ~2.25 GB active experts plus per-token routing overhead (full 23.73 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~88 tok/s

    MoE — speed based on ~3.1 GB active experts plus per-token routing overhead (full 32.6 GB loaded in VRAM)

  • Qwen3 Coder 30B-A3B 30.5B Great Tool use
    MMLU 81
    Q4_K_M
    Context 256K Speed ~120 tok/s

    MoE — speed based on ~1.7 GB active experts plus per-token routing overhead (full 17.28 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~90 tok/s

    MoE — speed based on ~2.98 GB active experts plus per-token routing overhead (full 30.25 GB loaded in VRAM)

  • Mistral Small 3.1 24B 23.6B Great VisionTool use
    MMLU 81
    Q4_K_M
    Context 128K Speed ~32 tok/s
    Q8_0
    Context 128K Speed ~19 tok/s
  • DeepSeek R1 Distill Qwen 32B 32.5B Great Reasoning
    MMLU 80
    Q4_K_M
    Context 128K Speed ~23 tok/s
    Q8_0
    Context 128K Speed ~14 tok/s
  • Qwen 2.5 14B 14.7B Great Tool use
    MMLU 79.9
    Q4_K_M
    Context 128K Speed ~49 tok/s
    Q8_0
    Context 128K Speed ~29 tok/s
  • Nemotron Cascade 2 30B A3B 30B Great ReasoningTool use
    MMLU 79.8
    Q4_K_M
    Context 256K Speed ~104 tok/s

    MoE — speed based on ~2.3 GB active experts plus per-token routing overhead (full 23.03 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~87 tok/s

    MoE — speed based on ~3.13 GB active experts plus per-token routing overhead (full 31.28 GB loaded in VRAM)

  • Llama 4 Scout 109B Great VisionTool use
    MMLU 79.6
    Q4_K_M
    Context 128K Speed ~39 tok/s

    MoE — speed based on ~9.49 GB active experts plus per-token routing overhead (full 60.87 GB loaded in VRAM)

    Q8_0 Doesn't fit · needs 106.9 GB
  • Qwen3.5 4B 4B Great VisionTool useReasoning
    MMLU 79.1
    Q4_K_M
    Context 256K Speed ~130 tok/s
    Q8_0
    Context 256K Speed ~89 tok/s
  • Qwen3 8B 8.2B Great ReasoningTool use
    MMLU 79
    Q4_K_M
    Context 128K Speed ~81 tok/s
    Q8_0
    Context 128K Speed ~50 tok/s
  • Qwen 2.5 Coder 32B 32.5B Great
    MMLU 79
    Q4_K_M
    Context 128K Speed ~23 tok/s
    Q8_0
    Context 128K Speed ~14 tok/s
  • Gemma 3 27B 27B Great Vision
    MMLU 78
    Q4_K_M
    Context 128K Speed ~28 tok/s
    Q8_0
    Context 128K Speed ~16 tok/s
  • Gemma 4 12B 11.96B Great VisionTool useReasoning
    MMLU 77.2
    Q4_K_M
    Context 256K Speed ~60 tok/s
    Q8_0
    Context 256K Speed ~36 tok/s
  • Nemotron Nano 12B 12.6B Great ReasoningTool use
    MMLU 76
    Q4_K_M
    Context 128K Speed ~57 tok/s
    Q8_0
    Context 128K Speed ~35 tok/s
  • GPT-OSS 20B 21B Great ReasoningTool use
    MMLU 75
    Q4_K_M
    Context 128K Speed ~116 tok/s

    MoE — speed based on ~1.86 GB active experts plus per-token routing overhead (full 10.83 GB loaded in VRAM)

    Q8_0
    Context 128K Speed ~114 tok/s

    MoE — speed based on ~1.93 GB active experts plus per-token routing overhead (full 11.28 GB loaded in VRAM)

  • Qwen 2.5 7B 7.62B Good Tool use
    MMLU 74.2
    Q4_K_M
    Context 128K Speed ~86 tok/s
    Q8_0
    Context 128K Speed ~54 tok/s
  • Qwen 2.5 VL 7B 7.62B Good VisionTool use
    MMLU 74.2
    Q4_K_M
    Context 128K Speed ~86 tok/s
    Q8_0
    Context 128K Speed ~54 tok/s
  • Gemma 3 12B 11.8B Good Vision
    MMLU 74
    Q4_K_M
    Context 128K Speed ~59 tok/s
    Q8_0
    Context 128K Speed ~36 tok/s
  • Qwen 2.5 Coder 14B 14.7B Good
    MMLU 74
    Q4_K_M
    Context 128K Speed ~49 tok/s
    Q8_0
    Context 128K Speed ~29 tok/s
  • Llama 3.2 Vision 11B 10.6B Good VisionTool use
    MMLU 73
    Q4_K_M
    Context 128K Speed ~66 tok/s
    Q8_0
    Context 128K Speed ~40 tok/s
  • DeepSeek R1 Distill Qwen 14B 14.7B Good Reasoning
    MMLU 72
    Q4_K_M
    Context 128K Speed ~49 tok/s
    Q8_0
    Context 128K Speed ~29 tok/s
  • Devstral Small 24B 23.6B Good Tool use
    MMLU 72
    Q4_K_M
    Context 128K Speed ~32 tok/s
    Q8_0
    Context 128K Speed ~19 tok/s
  • Phi-3 Mini 3.8B 3.82B Good
    MMLU 70.9
    Q4_K_M
    Context 128K Speed ~143 tok/s
    Q8_0
    Context 128K Speed ~96 tok/s
  • Phi-4 Mini 3.8B 3.84B Good
    MMLU 70
    Q4_K_M
    Context 128K Speed ~139 tok/s
    Q8_0
    Context 128K Speed ~96 tok/s
  • Llama 3.1 8B 8.03B Good Tool use
    MMLU 69.4
    Q4_K_M
    Context 128K Speed ~82 tok/s
    Q8_0
    Context 128K Speed ~51 tok/s
    fp16
    Context 128K Speed ~29 tok/s
  • Gemma 4 E4B 4.5B Good VisionTool useReasoning
    MMLU 69.4
    Q4_K_M
    Context 128K Speed ~76 tok/s
    Q8_0
    Context 128K Speed ~54 tok/s
  • Mistral Nemo 12B 12.2B Good Tool use
    MMLU 68
    Q4_K_M
    Context 128K Speed ~57 tok/s
    Q8_0
    Context 128K Speed ~35 tok/s
  • Qwen 2.5 Coder 7B 7.62B Good
    MMLU 68
    Q4_K_M
    Context 128K Speed ~86 tok/s
    Q8_0
    Context 128K Speed ~54 tok/s
  • Qwen 2.5 VL 3B 3.09B Fair Vision
    MMLU 65.6
    Q4_K_M
    Context 128K Speed ~166 tok/s
    Q8_0
    Context 128K Speed ~114 tok/s
  • Qwen 2.5 3B 3.09B Fair Tool use
    MMLU 65.6
    Q4_K_M
    Context 32K Speed ~166 tok/s
    Q8_0
    Context 32K Speed ~114 tok/s
  • Ministral 8B 8.02B Fair Tool use
    MMLU 65
    Q4_K_M
    Context 128K Speed ~76 tok/s
    Q8_0
    Context 128K Speed ~51 tok/s
  • SmallThinker 3B 3.09B Fair Reasoning
    MMLU 65
    Q4_K_M
    Context 32K Speed ~157 tok/s
    Q8_0
    Context 32K Speed ~107 tok/s
  • Llama 3.2 3B 3.21B Fair Tool use
    MMLU 63.4
    Q4_K_M
    Context 128K Speed ~161 tok/s
    Q8_0
    Context 128K Speed ~110 tok/s
    fp16
    Context 128K Speed ~66 tok/s
  • Gemma 3 4B 3.88B Fair Vision
    MMLU 63
    Q4_K_M
    Context 128K Speed ~139 tok/s
    Q8_0
    Context 128K Speed ~95 tok/s
  • Codestral 22B 22.2B Fair
    MMLU 62
    Q4_K_M
    Context 32K Speed ~34 tok/s
    Q8_0
    Context 32K Speed ~20 tok/s
  • Gemma 4 E2B 2.3B Fair VisionTool useReasoning
    MMLU 60
    Q4_K_M
    Context 128K Speed ~109 tok/s
    Q8_0
    Context 128K Speed ~82 tok/s
  • DeepSeek R1 Distill Qwen 7B 7.62B Fair Reasoning
    MMLU 58
    Q4_K_M
    Context 128K Speed ~86 tok/s
    Q8_0
    Context 128K Speed ~54 tok/s
  • Qwen3.5 2B 2B Fair VisionTool useReasoning
    MMLU 55.3
    Q4_K_M
    Context 256K Speed ~214 tok/s
    Q8_0
    Context 256K Speed ~162 tok/s
  • StarCoder2 15B 15.6B Basic
    MMLU 51.4
    Q4_K_M
    Context 16K Speed ~45 tok/s
    Q8_0
    Context 16K Speed ~27 tok/s
  • DeepSeek Coder 6.7B 6.74B Basic
    MMLU 50
    Q4_K_M
    Context 16K Speed ~96 tok/s
    Q8_0
    Context 16K Speed ~60 tok/s
  • Llama 3.2 1B 1.24B Basic Tool use
    MMLU 49.3
    Q4_K_M
    Context 128K Speed ~270 tok/s
    Q8_0
    Context 128K Speed ~210 tok/s
  • Qwen3.5 0.8B 0.8B Basic VisionTool useReasoning
    MMLU 29.7
    Q4_K_M
    Context 256K Speed ~317 tok/s
    Q8_0
    Context 256K Speed ~268 tok/s
  • Nemotron 3.5 Super VL 120B A12B 124.45B Basic VisionTool useReasoning
    MMLU
    Q4_K_M
    Context 1M Speed ~54 tok/s

    MoE — speed based on ~6.3 GB active experts plus per-token routing overhead (full 65.29 GB loaded in VRAM)

    Q8_0 Doesn't fit · needs 119.7 GB
  • K2-Horizon 7B 9B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 512K Speed ~74 tok/s
    Q8_0
    Context 512K Speed ~46 tok/s
  • Nanbeige4.3 33B 33.12B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 387K Speed ~22 tok/s
    Q8_0
    Context 319K Speed ~13 tok/s
  • Nemotron 3 Nano 4B 3.97B Basic ReasoningTool use
    MMLU
    Q4_K_M
    Context 256K Speed ~123 tok/s
    Q8_0
    Context 256K Speed ~93 tok/s
  • Ornith 1.5 9B 9.65B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~72 tok/s
    Q8_0
    Context 256K Speed ~45 tok/s
  • Ornith 1.5 35B A3B 35.95B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~121 tok/s

    MoE — speed based on ~1.69 GB active experts plus per-token routing overhead (full 20.22 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~91 tok/s

    MoE — speed based on ~2.94 GB active experts plus per-token routing overhead (full 35.21 GB loaded in VRAM)

  • Laguna XS 2.1 33.44B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~120 tok/s

    MoE — speed based on ~1.72 GB active experts plus per-token routing overhead (full 19.14 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~90 tok/s

    MoE — speed based on ~2.97 GB active experts plus per-token routing overhead (full 33.15 GB loaded in VRAM)

  • Nemotron 3.5 Bolt 74B A9B 74.89B Basic VisionTool useReasoning
    MMLU
    Q8_0
    Context 256K Speed ~42 tok/s

    MoE — speed based on ~8.83 GB active experts plus per-token routing overhead (full 73.44 GB loaded in VRAM)

  • Ornith 1.5 122B 125.09B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~59 tok/s

    MoE — speed based on ~5.65 GB active experts plus per-token routing overhead (full 70.64 GB loaded in VRAM)

    Q8_0 Doesn't fit · needs 123.5 GB
  • Flint 40B A2.5B 39.79B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~128 tok/s

    MoE — speed based on ~1.48 GB active experts plus per-token routing overhead (full 23.61 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~100 tok/s

    MoE — speed based on ~2.48 GB active experts plus per-token routing overhead (full 39.44 GB loaded in VRAM)

  • MiniCPM-V 4.7 35B A3B 35.21B Basic VisionTool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~121 tok/s

    MoE — speed based on ~1.68 GB active experts plus per-token routing overhead (full 19.71 GB loaded in VRAM)

    Q8_0
    Context 256K Speed ~91 tok/s

    MoE — speed based on ~2.93 GB active experts plus per-token routing overhead (full 34.37 GB loaded in VRAM)

  • MiMo V2.6 Distill Qwen 9B 9.41B Basic VisionTool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~71 tok/s
    Q8_0
    Context 256K Speed ~46 tok/s
  • LFM3 1.7B 1.73B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~232 tok/s
    Q8_0
    Context 256K Speed ~171 tok/s
  • Arete 27B 27.36B Basic VisionTool useReasoning
    MMLU
    Q4_K_M
    Context 256K Speed ~27 tok/s
    Q8_0
    Context 256K Speed ~16 tok/s
  • Muse Glimmer 30B 29.78B Basic VisionTool useReasoning
    MMLU
    Q4_K_M
    Context 128K Speed ~27 tok/s
    Q8_0
    Context 128K Speed ~16 tok/s
  • LFM2.5 2.6B 2.7B Basic Tool use
    MMLU
    Q4_K_M
    Context 128K Speed ~182 tok/s
    Q8_0
    Context 128K Speed ~126 tok/s
  • LFM2.5 24B A2B 23.97B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 128K Speed ~132 tok/s

    MoE — speed based on ~1.38 GB active experts plus per-token routing overhead (full 13.53 GB loaded in VRAM)

    Q8_0
    Context 128K Speed ~101 tok/s

    MoE — speed based on ~2.43 GB active experts plus per-token routing overhead (full 23.74 GB loaded in VRAM)

  • Mellum2.1 12B A2.5B Thinking 12.15B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 128K Speed ~125 tok/s

    MoE — speed based on ~1.55 GB active experts plus per-token routing overhead (full 7.52 GB loaded in VRAM)

    Q8_0
    Context 128K Speed ~100 tok/s

    MoE — speed based on ~2.48 GB active experts plus per-token routing overhead (full 12.04 GB loaded in VRAM)

  • LFM3-VL 2B 1.92B Basic VisionTool useReasoning
    MMLU
    Q4_K_M
    Context 128K Speed ~232 tok/s
    Q8_0
    Context 128K Speed ~171 tok/s
  • Nemotron 3.5 Bolt E4B 8B Basic Tool useReasoning
    MMLU
    Q4_K_M
    Context 64K Speed ~68 tok/s
    Q8_0
    Context 64K Speed ~54 tok/s

Tight fit 2

  • Gemma 2 27B 27.2B Great
    MMLU 75.2
    Q4_K_M
    Context 8K Speed ~28 tok/s

    Only 8K context — under the 16K needed to be practical

    Q8_0
    Context 8K Speed ~16 tok/s

    Only 8K context — under the 16K needed to be practical

  • Gemma 2 9B 9.24B Good
    MMLU 71.3
    Q4_K_M
    Context 8K Speed ~72 tok/s

    Only 8K context — under the 16K needed to be practical

    Q8_0
    Context 8K Speed ~45 tok/s

    Only 8K context — under the 16K needed to be practical

Notify me when new models fit my GPU

An email when a new model fits your 96 GB setup — no spam, unsubscribe anytime.

Doesn't fit 23

  • DeepSeek R1 0528 671B Excellent Reasoning
    MMLU 90.8
    Q4_K_M Needs 381.2 GB — 285.2 GB over
    Q8_0 Needs 664.4 GB — 568.4 GB over
  • MiMo V2.5 Pro 1020B Excellent Reasoning
    MMLU 89.4
    Q4_K_M Needs 586.5 GB — 490.5 GB over
    Q8_0 Needs 1013.0 GB — 917.0 GB over
  • MiniMax M2.1 229B Excellent ReasoningTool use
    MMLU 88
    Q4_K_M Needs 128.9 GB — 32.9 GB over
    Q8_0 Needs 226.5 GB — 130.5 GB over

Doesn't fit 23

  • DeepSeek R1 0528 671B Excellent Reasoning
    MMLU 90.8
    Q4_K_M Needs 381.2 GB — 285.2 GB over
    Q8_0 Needs 664.4 GB — 568.4 GB over
  • MiMo V2.5 Pro 1020B Excellent Reasoning
    MMLU 89.4
    Q4_K_M Needs 586.5 GB — 490.5 GB over
    Q8_0 Needs 1013.0 GB — 917.0 GB over
  • MiniMax M2.1 229B Excellent ReasoningTool use
    MMLU 88
    Q4_K_M Needs 128.9 GB — 32.9 GB over
    Q8_0 Needs 226.5 GB — 130.5 GB over
  • DeepSeek V4 Pro 1600B Excellent Tool useReasoning
    MMLU 87.5
    Q4_K_M Needs 900.1 GB — 804.1 GB over
    Q8_0 Needs 1670.1 GB — 1574.1 GB over
  • Qwen3.5 397B A17B 397B Excellent VisionTool useReasoning
    MMLU 87
    Q4_K_M Needs 227.4 GB — 131.4 GB over
    Q8_0 Needs 392.6 GB — 296.6 GB over
  • GLM-4.6 357B Excellent ReasoningTool use
    MMLU 87
    Q4_K_M Needs 200.8 GB — 104.8 GB over
    Q8_0 Needs 353.3 GB — 257.3 GB over
  • GLM 5.1 754B Excellent Tool use
    MMLU 86.9
    Q4_K_M Needs 428.2 GB — 332.2 GB over
    Q8_0 Needs 746.4 GB — 650.4 GB over
  • GLM 5.3 753.33B Excellent Tool useReasoning
    MMLU 86.8
    Q8_0 Needs 746.4 GB — 650.4 GB over
  • DeepSeek V4 Flash 284B Excellent Tool useReasoning
    MMLU 86.4
    Q4_K_M Needs 161.1 GB — 65.1 GB over
    Q8_0 Needs 282.1 GB — 186.1 GB over
  • GLM 5.3 Flash 321.32B Excellent Tool useReasoning
    MMLU 86.1
    Q4_K_M Needs 176.0 GB — 80.0 GB over
    Q8_0 Needs 310.4 GB — 214.4 GB over
  • Llama 4 Maverick 400B Excellent VisionTool use
    MMLU 85.5
    Q4_K_M Needs 226.3 GB — 130.3 GB over
    Q8_0 Needs 396.8 GB — 300.8 GB over
  • Step 3.5 Flash 197B Excellent ReasoningTool use
    MMLU 84
    Q4_K_M Needs 111.9 GB — 15.9 GB over
    Q8_0 Needs 195.2 GB — 99.2 GB over
    GPUs that can run this

    As an Amazon Associate, I earn from qualifying purchases.

  • GLM 5 753.86B Excellent Tool use
    MMLU 83
    Q4_K_M Needs 424.7 GB — 328.7 GB over
    Q8_0 Needs 746.4 GB — 650.4 GB over
  • MiMo V2.5 310B Great Reasoning
    MMLU 82.9
    Q4_K_M Needs 175.9 GB — 79.9 GB over
    Q8_0 Needs 306.8 GB — 210.8 GB over
  • MiniMax M2.5 228.7B Great ReasoningTool use
    MMLU 82
    Q4_K_M Needs 129.1 GB — 33.1 GB over
    Q8_0 Needs 226.7 GB — 130.7 GB over
  • Qwen3.8 Flash Next 180B Good VisionTool useReasoning
    MMLU 73.23
    Q4_K_M Needs 111.4 GB — 15.4 GB over
    Q8_0 Needs 175.4 GB — 79.4 GB over
    GPUs that can run this

    As an Amazon Associate, I earn from qualifying purchases.

  • Mistral Large 3 675B 673.42B Good VisionTool useReasoning
    MMLU 73.11
    Q4_K_M Needs 379.1 GB — 283.1 GB over
    Q8_0 Needs 666.6 GB — 570.6 GB over
  • Mistral Large 4 1T A52B 1049.43B Basic VisionTool useReasoning
    MMLU
    Q4_K_M Needs 591.3 GB — 495.3 GB over
    Q8_0 Needs 1038.6 GB — 942.6 GB over
  • Muse Spark Open 816B A42B 822.04B Basic VisionTool useReasoning
    MMLU
    Q4_K_M Needs 465.3 GB — 369.3 GB over
    Q8_0 Needs 811.9 GB — 715.9 GB over
  • Ling 3.1 Flash 572.12B Basic Tool useReasoning
    MMLU
    Q4_K_M Needs 333.4 GB — 237.4 GB over
    Q8_0 Needs 566.6 GB — 470.6 GB over
  • Beam 1.0 501B A23B 501.27B Basic Tool useReasoning
    MMLU
    Q4_K_M Needs 293.0 GB — 197.0 GB over
    Q8_0 Needs 496.2 GB — 400.2 GB over
  • Ornith 1.5 397B 403.4B Basic Tool useReasoning
    MMLU
    Q4_K_M Needs 227.6 GB — 131.6 GB over
    Q8_0 Needs 399.1 GB — 303.1 GB over
  • Step 3.7 Flash 198B Basic VisionTool useReasoning
    MMLU
    Q4_K_S Needs 104.0 GB — 8.0 GB over
    Q8_0 Needs 195.2 GB — 99.2 GB over
    GPUs that can run this

    As an Amazon Associate, I earn from qualifying purchases.