Local AI
Which local AI models fit your GPU?
Pick the memory on your graphics card. The table marks every build that fits, using the exact file sizes Hugging Face lists for 28 open models. Sizes read on November 16, 2026.
Sizes below are in GiB. With JavaScript on, each build is marked as fitting or not for the card you pick.
- Fits with the free room you asked for
- Loads, with less free room than you asked for
- File is larger than the card: part of it would run on the CPU
| Model | Q3_K_M | Q4_K_M | Q5_K_M | Q6_K | Q8_0 |
|---|---|---|---|---|---|
| Qwen3.5-2B | 1.03 | 1.19 | 1.34 | 1.47 | 1.87 |
| Phi-4-mini-instruct | 1.97 | 2.32 | 2.65 | 2.94 | 3.80 |
| Qwen3.5-4B | 2.14 | 2.55 | 2.93 | 3.28 | 4.17 |
| gemma-4-E2B-it | 2.36 | 2.89 | 3.13 | 4.19 | 4.70 |
| granite-4.0-h-tiny | 3.12 | 3.94 | 4.61 | 5.32 | 6.88 |
| Qwen2.5-Coder-7B-Instruct | 3.55 | 4.36 | 5.07 | 5.82 | 7.54 |
| gemma-4-E4B-it | 3.78 | 4.64 | 5.11 | 6.59 | 7.63 |
| LFM2.5-8B-A1B | No build listed- | 4.80 | 5.62 | 6.48 | 8.39 |
| Qwen3.5-9B | 4.35 | 5.29 | 6.13 | 6.95 | 8.87 |
| gemma-4-12B-it | 5.30 | 6.63 | 7.84 | 9.11 | 11.80 |
| Apriel-1.6-15b-Thinker | 6.65 | 8.18 | 9.56 | 11.03 | 14.29 |
| Qwen2.5-Coder-14B-Instruct | 6.84 | 8.37 | 9.79 | 11.29 | 14.62 |
| gpt-oss-20b | 10.72 | 10.83 | 10.91 | 11.21 | 11.28 |
| Mistral-Small-3.2-24B-Instruct-2506 | 10.69 | 13.35 | 15.61 | 18.02 | 23.33 |
| Devstral-Small-2-24B-Instruct-2512 | 10.69 | 13.35 | 15.61 | 18.02 | 23.33 |
| Qwen3.8-27B | No build listed- | 15.43 | 18.51 | 20.47 | 27.05 |
| Qwen3.5-27B | 12.58 | 15.59 | 18.26 | 20.91 | 26.63 |
| Qwen3.6-27B | 12.65 | 15.66 | 18.17 | 20.98 | 26.63 |
| gemma-4-26B-A4B-it | 11.85 | 15.78 | 19.70 | 21.58 | 25.02 |
| GLM-4.7-Flash | 13.61 | 17.05 | 19.94 | 23.00 | 29.66 |
| Qwen3-Coder-30B-A3B-Instruct | 13.70 | 17.28 | 20.23 | 23.37 | 30.25 |
| granite-4.0-h-small | 14.31 | 18.14 | 21.30 | 24.65 | 31.91 |
| Qwen2.5-Coder-32B-Instruct | 14.84 | 18.49 | 21.66 | 25.04 | 32.43 |
| KAT-Coder-V2.5-Dev | 15.11 | 19.92 | 23.30 | 27.99 | 34.38 |
| Qwen3.5-35B-A3B | 15.23 | 20.50 | 24.45 | 26.87 | 34.37 |
| Qwen3.6-35B-A3B | 15.46 | 20.61 | 24.64 | 27.30 | 34.37 |
| Qwen3-Coder-Next | 35.69 | 45.20 | 52.94 | 61.11 | 78.99 |
| gpt-oss-120b | 58.33 | 58.46 | 58.57 | 58.94 | 59.03 |
Sizes in GiB (1 GiB = 1,073,741,824 bytes). A dash means the repository lists no single build at that quantisation. Each model name links to the repository the sizes were read from.
How the calculator decides
A build is marked as fitting when its file size, plus the free room you chose, is no larger than the card's memory. Nothing on this page was run or timed. It is arithmetic on published file sizes.
- Card memory is counted in GiB. A card sold as 12 GB holds 12 GiB, which is 12,884,901,888 bytes. Hugging Face shows file sizes in decimal GB, so a file listed as 12.9 GB is 12.0 GiB. The table converts everything to GiB so the two can be compared.
- The free room is your estimate, not a measurement. A loaded model needs memory beyond its file: the context cache grows with the context length you set, and the desktop and browser use some of the card too. 1.5 GiB is a starting point for a short context. Raise it for long documents or a large context window. How much context a local model can hold explains what drives that number.
- A build that does not fit can still run. llama.cpp, Ollama and LM Studio can keep some layers in system memory and run them on the CPU. It works, and it is slower. See local AI without a GPU for what to expect.
- Vision needs a second file. Models with image input ship a separate projector file that loads next to the weights. Tick the box to add it.
- Lower quantisation is smaller and less exact. Q4_K_M is the usual default. What quantization costs you covers the trade.
Where the numbers come from
File sizes are the byte counts in each repository's file listing on Hugging Face. Parameter counts and licences are read from the original model's entry. All were read on November 16, 2026. Publishers sometimes replace a file in place, so check the repository before a large download.
Which model to pick once you know what fits
Fitting is the first filter, not the answer. The guides below say which model is the pick at each memory size, and why.