Local AI by VRAM Tier - 8GB | 12GB | 16GB | 24GB | 32GB
Deep dives: Chat | Coding | Translation | Vision | Speech | Agents
GitHub Copilot costs $19/month, sends your code to Microsoft’s servers, and still gets things wrong. Running a coding model locally costs nothing per query, keeps your proprietary code on your machine, and - depending on your GPU - gets surprisingly close in quality.
Updated August 2026. Refreshed monthly as models and runtimes change.
What changed this month: Kwaipilot’s KAT-Coder-V2.5-Dev joins the 16-24GB tier as a new Apache-2.0 agentic coder, and every citation has been moved onto model cards, lab repositories and official runtime listings. The rest of the lineup was re-verified and is unchanged: Qwen2.5-Coder for autocomplete, Qwen3.6 for agentic work, Devstral Small 2 and Qwen3-Coder-Next as the permissive alternatives.
The question is which model to run. There are two distinct jobs: autocomplete (fill-in-the-middle / FIM, the inline suggestions as you type) and agentic coding (asking questions, debugging, and having the model actually resolve issues across files). Some models do both. Most do one well.
The Benchmarks That Matter
- HumanEval - generate a Python function from a docstring. The classic test; most strong models now score 88%+
- SWE-bench Verified - fix a real GitHub issue autonomously. The closest thing to actual software engineering, and the benchmark that separates toys from tools
- FIM support - does the model support fill-in-the-middle for autocomplete? Not all chat models do
- Context window - how much of your codebase you can paste in at once
- License - whether you can use it in commercial work
Two grounding notes before the tables. SWE-bench ships in several variants - Verified, Multilingual and Pro among them - and they are different benchmarks with different task sets, so a score on one cannot be compared against a score on another. Every column below stays inside a single variant. And every VRAM figure is the static weight size at the named quantization, excluding the KV cache, which grows with context length.
8GB VRAM {#8gb}
GPUs: RTX 4060, RTX 3060 8GB, RTX 3070
Candidates
| Model | Params | HumanEval | FIM | VRAM (Q4 weights) | License |
|---|---|---|---|---|---|
| Qwen2.5-Coder 7B | 7B | 88.4% | Yes | ~5 GB | Apache 2.0 |
| DeepSeek Coder V2 Lite | 16B / 2.4B active (MoE) | 81.1% | Yes | ~5 GB | DeepSeek license |
For autocomplete: Qwen2.5-Coder 7B
The Qwen2.5-Coder 7B is still the best small autocomplete model. At 88.4% HumanEval in the technical report it beats far larger older models, supports FIM natively, handles 128K context, and covers 92 programming languages. At ~5GB it leaves 3GB free on an 8GB card. Plug it into Continue or Tabby for inline suggestions that rival Copilot on common patterns.
Budget MoE: DeepSeek Coder V2 Lite - 16B total but only 2.4B active, so it runs light (~5GB), supports FIM, and scores 81.1% HumanEval. Less community tooling than Qwen, but a solid free alternative.
The honest take: 8GB coding models handle autocompletion well but struggle with complex refactoring, multi-file changes, and large-codebase understanding. Use them for the routine 80% and reach for a bigger model or the cloud for the hard 20%.
For all use cases at this level, see the 8GB VRAM complete guide.
12GB VRAM {#12gb}
GPUs: RTX 3060 12GB, RTX 4070
Winner: Qwen2.5-Coder 14B
At 89.6% HumanEval in the technical report, with FIM support and 128K context, the Qwen2.5-Coder 14B is the pick for this tier. It fits in ~9GB, leaving ~3GB for context. It handles multi-language projects, framework-specific patterns, and generates tests alongside implementations - and it remains the strongest dedicated autocomplete model that fits a 12GB card.
For agentic use (running the model through a tool that edits files), 12GB is tight; the models that resolve real issues well start at the 16GB tier below.
For all use cases at this level, see the 12GB VRAM complete guide.
16GB VRAM {#16gb}
GPUs: RTX 4060 Ti 16GB, RTX 5060, Intel Arc A770, AMD RX 7800 XT
Candidates
| Model | Params | SWE-bench Verified (self-reported) | FIM | VRAM (Q4 weights) | License |
|---|---|---|---|---|---|
| Qwen3.6-35B-A3B | 35B / 3B active (MoE) | 73.4% | No | per Ollama tag list | Apache 2.0 |
| KAT-Coder-V2.5-Dev | 35B / 3B active (MoE) | 69.40% | No | not published | Apache 2.0 |
| Qwen2.5-Coder 14B | 14B | (autocomplete model) | Yes | ~9 GB | Apache 2.0 |
Winner (agentic): Qwen3.6-35B-A3B
Qwen3.6-35B-A3B is the default here: an Apache-2.0 mixture-of-experts model that self-reports 73.4% on SWE-bench Verified, meaning it resolves real GitHub issues rather than just completing lines. VRAM is the catch. A 35B-class MoE keeps all experts resident, so it does not fit a 16GB card at Q4 and you will be running an aggressive quant with modest context; read the exact tag size from Ollama’s Qwen3.6 listing before committing. If it is too tight, drop to the 24GB tier’s dense options or stay on Qwen2.5-Coder 14B for autocomplete.
New this month: Kwaipilot KAT-Coder-V2.5-Dev
KAT-Coder-V2.5-Dev is the one genuinely new local coding model since the July pass. It is Apache-2.0, a 35B-total / 3B-active mixture-of-experts with a 262K context window, with its Hugging Face repository created 2026-07-23 and last updated 2026-07-28 (the model card itself states no release day). It is the text-only variant of KAT-Coder-V2.5, so if you serve it through vLLM or SGLang you need to pass --language-model-only, because the parent model carries a vision tower. Hugging Face lists more than 50 community quantizations of it for llama.cpp, LM Studio, Jan and Ollama; the model card itself does not mention quantized builds.
Its self-reported scores, kept in separate variants because they are not comparable to each other:
| Benchmark | Score (self-reported) |
|---|---|
| SWE-bench Verified | 69.40% |
| SWE-bench Multilingual | 63.00% |
| SWE-bench Pro | 45.96% |
| Terminal-Bench 2.1 | 41.02% |
Comparing it with Qwen3.6-35B-A3B is harder than it looks. Qwen reports 73.4% for its own model on SWE-bench Verified, while KAT re-ran that same model on its own harness and got 64.40%, behind KAT’s 69.40%, footnoting “an approximate 10 pp gap compared with the official results” from harness versions and test-set fixes. The two labs’ headline numbers disagree by about ten points on the same model and the same benchmark, so no ordering between them is claimed here. It is the same size class and the same MoE shape, which puts it squarely in the 16-24GB bracket alongside the Qwen model. No VRAM figure is published for it, so budget from the quantized build you actually download rather than from a number quoted anywhere, including here. The Apache-2.0 licence and the 262K context are the reasons to try it over the Qwen model rather than the benchmark line.
Autocomplete on the same card: keep Qwen2.5-Coder 14B (~9GB) installed for fast FIM inline suggestions and switch to the bigger model for agentic sessions.
For all use cases at this level, see the 16GB VRAM complete guide.
24GB VRAM {#24gb}
GPUs: RTX 3090, RTX 4090
This tier is where local coding genuinely competes with cloud assistants.
Candidates
| Model | Params | HumanEval | SWE-bench Verified (self-reported) | FIM | VRAM (Q4 weights) | License |
|---|---|---|---|---|---|---|
| Qwen3.6-27B | 27B dense | - | 77.2% | No | ~17 GB | Apache 2.0 |
| Qwen2.5-Coder 32B | 32B | 92.7% | - | Yes | ~20 GB | Apache 2.0 |
| Devstral Small 2 | 24B dense | - | 68.0% | No | 14.3 GB | Apache 2.0 (deprecated 2026-02-27) |
| KAT-Coder-V2.5-Dev | 35B / 3B active (MoE) | - | 69.40% | No | not published | Apache 2.0 |
For agentic coding: Qwen3.6-27B
The Qwen3.6-27B dense model self-reports 77.2% on SWE-bench Verified and comes in around 17GB of Q4 weights, leaving context headroom on a 24GB card. Run it through Aider or OpenHands and it resolves real issues autonomously. This is the strongest agentic coder that fits a single 24GB GPU.
For autocomplete: Qwen2.5-Coder 32B
92.7% HumanEval with FIM, 128K context, and 92 programming languages at ~20GB. Still the best local autocomplete model at any tier - the reason a lot of RTX 4090 owners cancelled Copilot.
Apache-licensed alternatives: Devstral Small 2 from Mistral is 24B dense, Apache 2.0, 256K context, 68.0% SWE-bench Verified, and 14.3GB at Q4_K_M. The long context and permissive license make it a strong agentic pick when you want room for big repos. One thing to know before you build on it: Mistral marks this model deprecated as of 2026-02-27 and points new integrations at Mistral Medium 3.5, which is a hosted model rather than an open-weight replacement. The Apache-2.0 weights stay downloadable and keep working, but nobody is maintaining them. KAT-Coder-V2.5-Dev is the other one: also Apache 2.0, 262K context, 69.40% SWE-bench Verified self-reported, with no published VRAM figure but a 3B-active MoE design that should run comfortably on this tier at a mid-range quant.
For all use cases at this level, see the 24GB VRAM complete guide.
32GB VRAM {#32gb}
GPUs: RTX 5090
Winner: Qwen2.5-Coder 32B (Q6) for autocomplete, Qwen3.6-27B for agentic
At 32GB you can run Qwen2.5-Coder 32B at Q6_K (27GB) for the cleanest autocomplete - fewer hallucinated API calls than Q4 - or run Qwen3.6-27B with maximum context for agentic work. You can also comfortably run Devstral Small 2 at Q8_0 (25.1GB) if you prefer its 256K context and Apache license, keeping its deprecated status in mind, or KAT-Coder-V2.5-Dev at a higher quant than a 16GB card allows.
The stretch: Qwen3-Coder-Next is an 80B / 3B-active MoE, Apache 2.0, 262K context. Its model card publishes benchmarks only as images, and no machine-readable SWE-bench figure could be traced to a Qwen source for this update, so no score is quoted here. It is the most capable open coding model here, but be realistic about VRAM: an 80B mixture-of-experts keeps every expert resident regardless of how few are active per token, so a single 32GB card means a low quant or CPU offload and reduced speed. Impressive, but not a smooth interactive experience yet.
For all use cases at this level, see the 32GB VRAM complete guide.
Cross-Tier Summary
All SWE-bench figures in this table are SWE-bench Verified and self-reported by the model’s own lab. HumanEval figures are shown separately and are not comparable to them.
| Tier | Autocomplete Pick | Agentic Pick | Key figure |
|---|---|---|---|
| 8GB | Qwen2.5-Coder 7B | - | 88.4% HumanEval |
| 12GB | Qwen2.5-Coder 14B | - | 89.6% HumanEval |
| 16GB | Qwen2.5-Coder 14B | Qwen3.6-35B-A3B (alt: KAT-Coder-V2.5-Dev) | 73.4% / 69.40% SWE-bench Verified |
| 24GB | Qwen2.5-Coder 32B | Qwen3.6-27B (alt: Devstral Small 2, KAT-Coder-V2.5-Dev) | 92.7% HumanEval, 77.2% SWE-bench Verified |
| 32GB | Qwen2.5-Coder 32B (Q6) | Qwen3.6-27B | 92.7% HumanEval, 77.2% SWE-bench Verified |
IDE Integration
Autocomplete with Continue (VS Code / JetBrains)
# Pull your autocomplete model
ollama pull qwen2.5-coder:7b # 8GB tier
ollama pull qwen2.5-coder:14b # 12-16GB tier
ollama pull qwen2.5-coder:32b # 24GB+ tierInstall Continue in VS Code or JetBrains, point it at your local Ollama server (http://localhost:11434), and enable tab autocomplete. The experience is close to Copilot. Full walkthrough: Self-Host Code Completion with Continue + Ollama.
Autocomplete with Tabby
Tabby is purpose-built for self-hosted completion - model management, codebase indexing for context-aware suggestions, and plugins for VS Code, JetBrains, and Vim. Setup: Self-Host Tabby AI Code Completion.
Agentic coding with Aider
pip install aider-chat
aider --model ollama/qwen3.6:27bAider sends your files to the local model, gets proposed changes, and applies them with git integration. At a self-reported 77.2% SWE-bench Verified, Qwen3.6-27B resolves real issues, not just completions. Check the Ollama library for the exact current model tags.
What Local Coding Models Can’t Do (Yet)
- Full repository understanding - they see what you paste in, not your entire codebase. Indexing (Tabby) helps but is not the same as a cloud IDE agent
- Complex multi-file refactoring - models below the 27B tier struggle to stay consistent across many files
- Reliable test generation - they generate tests, but the tests often do not compile or test the wrong thing. Always review
- Unfamiliar frameworks - obscure libraries get hallucinated APIs
The 24GB+ tier with Qwen3.6-27B and Qwen2.5-Coder 32B genuinely threatens paid coding assistants for most workflows. Below that, local models are best as a supplement to your own expertise rather than a replacement for it.
All eleven local-AI guides, by hardware tier and by task, are indexed at Local AI by VRAM: Which Models Fit Your GPU.