Open dataset: LLM memory needs and GPUs
The data behind this site, as two JSON files: 214 model rows (107 models, one row per quantization) and 132 GPUs. MIT licensed, updated 2026-11-14.
What it’s for
Answering “will this model fit on that GPU, and with how much context?” needs two numbers per model that are hard to find in one place: the real size of the quantized weights, and how much memory each token of context costs. This dataset has both for every model the site tracks, measured the same way, so you can use them in your own tool, script or comparison.
Memory needed at a context of C thousand tokens is weight_gb + kv_per_1k_gb × min(C, max_context_k). The site adds about 1 GB of runtime
overhead when estimating speed, and treats under 16K of usable context as a tight fit.
How it’s produced
- Weights are the actual GGUF file sizes on Hugging Face (from the model’s official GGUF repo or a widely used quantizer such as bartowski or unsloth), summed across shards. Vision projectors (mmproj), multi-token-prediction and draft-model files are left out.
- KV cache is computed from the model’s
config.json:2 × layers × kv_heads × head_dim × 2 bytes × 1000 ÷ 2³⁰for fp16 keys and values. Only layers whose cache grows with the context count: full-attention layers (and DeepSeek-style sparse attention). Sliding-window, linear-attention and convolution layers in hybrid models hold a fixed-size state and are left out. Multi-head latent attention models (DeepSeek, GLM-5, Mistral Large 3) store one compressed latent per token per layer, so there is no ×2 for keys and values. - Mixture-of-experts models carry
active_weight_gb: the weights scaled by active ÷ total parameters, which is what each generated token has to read, and so what sets speed. All ofweight_gbstill has to fit in memory. - Benchmarks are taken from model cards and public leaderboards: MMLU-Pro for
mmlu_scoreand SWE-bench Verified forswe_bench_score. They are never estimated; a missing score is null. - GPUs: memory and bandwidth from manufacturer specifications and datasheets.
Rows are added by a script that flags weight sizes far from what the parameter count and quantization imply (the usual sign of a missing shard or a stray auxiliary file), and every release passes the same validation tests that guard the site, such as identical features across a model’s quantizations and lighter quantizations being smaller. The scripts that build the data are in the GitHub repo.
Format
Each file is one JSON object: a small envelope, then the rows under models or gpus.
{
"schema_version": 1,
"updated": "2026-11-14",
"license": "MIT",
"source": "https://whatmodelscanirun.com",
"docs": "https://whatmodelscanirun.com/data/",
"count": 214,
"models": [ … ]
} Rows are sorted by id and every row has every field, with null where a value doesn’t
apply, so a diff between two releases shows only what changed. GB means 2³⁰ bytes throughout.
Model fields
| Field | Type | Meaning |
|---|---|---|
id | string | Stable row id, one per model per quantization (e.g. llama-3.1-8b-q4). Never reused for a different row. |
name | string | Model name, shared by all quantizations of the same model. Group rows on it. |
url | string | The model’s page on this site (all quantizations share one page). |
quantization | string | GGUF quantization: Q4_K_M, Q4_K_S, Q8_0 or fp16. |
params_b | number | Total parameters, in billions. |
weight_gb | number | Size of the GGUF weights in GB (2³⁰ bytes), summed across shards. Excludes vision projectors (mmproj), MTP and draft heads. |
gguf_repo | string | null | Hugging Face repo (owner/name) holding this row’s GGUF, checked against weight_gb. Runs as ollama run hf.co/<gguf_repo>:<quant> or llama-server -hf <gguf_repo>:<quant> (F16 for fp16). null when no source could be verified. |
gguf_shards | number | null | How many files the GGUF is split into (1 = a single file; Ollama can’t pull split GGUFs from Hugging Face). null when gguf_repo is null. |
active_weight_gb | number | null | Mixture-of-experts only: the weights read per token (total weights scaled by active ÷ total parameters), in GB. null for dense models. |
kv_per_1k_gb | number | fp16 KV cache per 1,000 tokens of context, in GB. Counts only layers whose cache grows with context (see below). |
max_context_k | number | Trained context length in units of 1,024 tokens (128 = 131,072 tokens). |
layers | number | Total transformer layers, of every kind. Used to estimate partial offload to system RAM. |
features | string[] | Sorted subset of vision, reasoning, tool_use. Identical across a model’s quantizations. |
mmlu_score | number | null | MMLU-Pro (%) for current entries; some early rows carry original MMLU. null when not published. |
swe_bench_score | number | null | SWE-bench Verified (%). null when not published. Never SWE-bench Pro or other variants. |
notes | string | Free-text architecture and sourcing notes. For people, not parsers: wording changes between releases. |
GPU fields
| Field | Type | Meaning |
|---|---|---|
id | string | Stable kebab-case id (e.g. rtx-4090), also the page slug. |
name | string | Product name (e.g. GeForce RTX 4090, M4 Max). |
url | string | The GPU’s page on this site. |
manufacturer | string | NVIDIA, AMD, Intel or Apple. |
vram_gb | number | null | Memory in GB, for discrete cards. null for unified-memory parts. |
bandwidth_gbps | number | null | Peak memory bandwidth in GB/s, for discrete cards. null for unified-memory parts. |
vram_options | object[] | null | Unified-memory parts (Apple Silicon, AMD Ryzen AI Max, RTX Spark): one { vram_gb, bandwidth_gbps } per memory configuration, smallest first. null for discrete cards. |
msrp_usd | number | null | Launch price in US dollars, where known. |
Examples
curl and jq
curl -s https://whatmodelscanirun.com/data/models.json \
| jq '.models[] | select(.name == "Llama 3.1 8B") | {quantization, weight_gb, kv_per_1k_gb}' JavaScript
const res = await fetch('https://whatmodelscanirun.com/data/models.json');
const { models } = await res.json();
// VRAM for a model at a given context: weights + KV cache
const vramFor = (m, contextK) => m.weight_gb + m.kv_per_1k_gb * Math.min(contextK, m.max_context_k);
const fits24 = models.filter((m) => vramFor(m, 16) <= 24);
console.log(fits24.map((m) => `${m.name} ${m.quantization}`)); Python
import json, urllib.request
url = "https://whatmodelscanirun.com/data/models.json"
models = json.load(urllib.request.urlopen(url))["models"]
vram_gb, context_k = 24, 32
for m in models:
need = m["weight_gb"] + m["kv_per_1k_gb"] * min(context_k, m["max_context_k"])
if need <= vram_gb:
print(f'{m["name"]} {m["quantization"]}: {need:.1f} GB {m["url"]}') Updates and versioning
The data is refreshed about every two weeks, and never more than a month apart; updated is the
date of the release. URLs stay the same, so fetch them fresh rather than caching for long.
schema_version is 1. Additive changes (new rows, new fields, new values in features or quantization) keep the version, so ignore fields you don’t know.
Breaking changes (renaming or removing a field, or changing its unit or meaning) bump it. Corrections to
values are ordinary updates.
Licence and attribution
The dataset is MIT licensed, like the site’s code.
Use it in anything, commercial or not. Attribution is appreciated: a link to whatmodelscanirun.com (or to a row’s url) helps other people find the
full tool.
Spotted a wrong number? Open an issue with the source and it will be fixed in the next release.