Local AI by VRAM Tier - 8GB | 12GB | 16GB | 24GB | 32GB
Deep dives: Chat | Coding | Translation | Vision | Speech | Agents
August 2026 refresh: this page was a full model generation behind the rest of the series. The chat, coding, agent and vision picks now match it. Vision moves to Qwen3.5-9B, and every VRAM number below is a static weight size with the quantisation named.
12GB is where local AI goes from usable to genuinely good. A 9B model runs with real context headroom, two smaller models fit at once, and a full speech pipeline runs alongside a chat model. What changed this year is not the tier, it is the models that fill it: the Qwen line moved from Qwen 3 to Qwen3.5, and the vision and text families merged.
Updated August 2026. Refreshed monthly as models and runtimes change.
Your Hardware
12GB VRAM cards: RTX 3060 12GB and RTX 4070. The RTX 3060 12GB is the long-standing value option because it puts a full 12GB behind a mid-range card, though street prices move constantly and are worth checking before you buy.
How to read the VRAM numbers on this page. Every figure is the static size of the model weights at the named quantisation, taken from published GGUF file sizes, with two exceptions that are labelled where they appear: the Whisper figures are OpenAI’s own required-VRAM numbers rather than weight sizes, and the Step-Audio-EditX figures are StepFun’s stated GPU memory rather than weight sizes. The KV cache is allocated on top and grows with context length, so treat each number as a floor, not a total. Anything within 1-2GB of the 12GB ceiling fits with minimal context room.
The rough math: at Q4_K_M a model needs roughly 0.6-0.8GB per billion parameters, so 12GB comfortably holds the 9-14B class at Q4. Q8_0 on a 9B fits, but leaves little room for context.
Quick Reference
| Use Case | Best Pick | Static weights | Quantisation | Note |
|---|---|---|---|---|
| Chat | Qwen3.5-9B | 5.68 GB | Q4_K_M | Apache-2.0, generous context room |
| Chat (reasoning) | Apriel-1.6-15B-Thinker | 8.79 GB | Q4_K_M | MIT, fits with context room |
| Coding (autocomplete) | Qwen2.5-Coder 14B | ~9 GB | Q4_K_M | FIM, 128K context |
| Coding (chat) | Qwen3.5-9B | 5.68 GB | Q4_K_M | Current Qwen generation |
| Translation | TranslateGemma 12B | 8.1 GB | Ollama build | Custom Gemma terms |
| Vision | Qwen3.5-9B + mmproj | 10.45 GB | Q8_0 + mmproj-F16 | Minimal context room |
| Speech (STT) | Parakeet TDT 0.6B v3 | ~2 GB | - | English plus 24 languages |
| Speech (TTS) | Step-Audio-EditX (AWQ) | ~6-8 GB | AWQ 4-bit | Apache-2.0 |
| Agents | Qwen3.5-9B | 5.68 GB | Q4_K_M | Or Apriel-1.6-15B for planning |
Chat & General Assistant
The 12GB chat pick is Qwen3.5-9B, Apache-2.0 and ungated. Static weights are 5.68GB at Q4_K_M and 9.53GB at Q8_0. The Q4_K_M build leaves the most room for long conversations; Q8_0 fits but with minimal context room once the KV cache is allocated.
This replaces the Qwen 3 14B recommendation that stood here. Qwen3.5 is a newer generation, and unlike Qwen 3 it is a unified vision-language model at every dense size, so one download covers both text and images.
Reasoning alternative: Apriel-1.6-15B-Thinker, a 15B dense MIT-licensed model with 131K context. ServiceNow publishes a first-party GGUF: 8.79GB at Q4_K_M, plus a 0.87GB vision projector if you want its image support. That leaves roughly 3GB of context room on a 12GB card.
Full comparison: local chat model guide.
Coding
Autocomplete: Qwen2.5-Coder 14B at roughly 9GB, with fill-in-the-middle and 128K context. It is an older generation than the chat pick above and we are keeping it deliberately: the newer Qwen releases are general models without dedicated FIM training, so the Qwen2.5-Coder family is still the strongest inline-completion option at this tier. The 7B build at roughly 5GB is the choice if you want a second model resident alongside it.
Chat-based coding: Qwen3.5-9B, the same download as your chat model. For code questions, refactor suggestions and explanations, the current-generation general model beats an older coder-specific one.
Agentic coding does not fit here. The models that resolve real GitHub issues across files start at 16GB and are comfortable at 24GB. At 12GB, keep autocomplete local and reach up a tier for autonomous work.
Full comparison: local coding model guide.
Translation
TranslateGemma 12B is listed on Ollama at 8.1GB, leaving room to hold a long document in context and translate it in one pass. Its technical report puts it at MetricX 3.60 and COMET22 83.5, ahead of the much larger Gemma 3 27B baseline it was measured against. Those figures come from the model’s own authors.
One licensing point that gets glossed over: TranslateGemma ships under Google’s custom Gemma terms with a prohibited-use policy attached, not a blanket open licence. Read them before building a product on it. The Hugging Face repositories are also gated and need a manual access request; the Ollama tags are not.
Full comparison: local translation model guide.
Vision
This is the biggest change on the page. The old recommendation was Qwen3-VL 8B. That family has had no new release since Qwen3.5 shipped: no generative Qwen3-VL repository has been updated since November 2025, while the first Qwen3.5 repository appeared in February 2026. Only the Qwen3-VL embedding and reranker repositories have been touched since.
The pick: Qwen3.5-9B with its vision projector. At Q8_0 the weights are 9.53GB and the mmproj-F16 file adds 0.92GB, for 10.45GB total. That fits a 12GB card with roughly 1.5GB left, which is minimal context room, so budget accordingly. If you need long prompts or a second resident model, the Q4_K_M build plus mmproj is 6.60GB.
Alibaba’s own card states that early-fusion multimodal training “achieves cross-generational parity with Qwen3 and outperforms Qwen3-VL models across reasoning, coding, agents, and visual understanding benchmarks.” The supporting numbers, comparing Qwen3.5-9B against the larger Qwen3-VL-30B-A3B, are self-reported by Alibaba on its own model cards:
| Benchmark | Qwen3.5-9B | Qwen3-VL-30B-A3B |
|---|---|---|
| MMMU | 78.4 | 76.0 |
| MMMU-Pro | 70.1 | 63.0 |
| MathVision | 78.9 | 65.7 |
| MathVista (mini) | 85.7 | 81.9 |
| RealWorldQA | 80.3 | 77.4 |
| MMStar | 79.7 | 75.5 |
| OCRBench | 89.2 | 83.9 |
| OmniDocBench 1.5 | 87.7 | 86.8 |
| CC-OCR | 79.3 | 77.8 |
Treat that table with the scepticism it deserves. Both product lines belong to the same vendor, so this is Alibaba grading its own successor against its own predecessor. We looked for independent confirmation and found none: the OpenCompass OpenVLM leaderboard is JavaScript-only and returned no data, and Artificial Analysis runs no vision benchmark for either family.
Two practical caveats. Alibaba publishes no official GGUF conversion for Qwen3.5, only for Qwen3-VL, so every Qwen3.5 GGUF you can download is third-party work such as the Unsloth builds. Runtime support, though, is in place rather than promised: llama.cpp PR #19468 merged in February 2026, verified against Qwen Team preview checkpoints “in both vision and pure text modes (with or without mmproj file)”, and the official Ollama library carries tags from 0.8b to 122b marked “Text, Image”.
Full comparison: local vision model guide.
Speech
12GB is where simultaneous transcription and speech synthesis becomes comfortable.
Transcription: Parakeet TDT 0.6B v3 loads in about 2GB and covers English plus 24 more languages, leaving most of the card free. Whisper large-v3-turbo at roughly 6GB is the option for broader language coverage. We are not restating an accuracy ranking between them this month: the individual models are unchanged, but the live Open ASR leaderboard order could not be reliably reconfirmed in this pass.
If you transcribe through whisper.cpp, update it. The project is at v1.9.2 (4 August 2026); v1.9.0 (17 June 2026) added native NVIDIA Parakeet support, so one runtime now handles both model families.
Text-to-speech: Step-Audio-EditX at roughly 6-8GB as an AWQ 4-bit build is the quality pick, and it is Apache-2.0 so it is usable commercially. Chatterbox (MIT, roughly 4-6GB) is the lighter voice-cloning option, and Kokoro-82M (Apache-2.0, under 1GB, CPU-capable) is the choice when you want TTS to cost almost nothing.
Best combination here: Parakeet (~2GB) plus Step-Audio-EditX AWQ (~6-8GB) is roughly 8-10GB together, a full commercial-grade voice loop with room left over.
Full comparison: local speech model guide.
Agents
Qwen3.5-9B at 5.68GB is the default agent model here, and the headroom is the point: agents pull large tool responses (database rows, page contents) into context, and at this tier you can afford them. Its tool-call format parses cleanly across the common frameworks.
When the hard part is deciding which tool to call rather than how, step up to Apriel-1.6-15B-Thinker. Its reasoning training helps with multi-step planning, and at 8.79GB it leaves roughly 3GB for the KV cache on a 12GB card.
Full comparison: local agent model guide.
Getting Started
# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh
# Pull the essentials for 12GB
ollama pull qwen3.5:9b # Chat + coding chat + vision + agents
ollama pull qwen2.5-coder:14b # Autocomplete
ollama pull translategemma:12b # Translation
# Start chatting
ollama run qwen3.5:9bThe qwen3.5 tags carry “Text, Image” capabilities, so the same pull covers vision. Model tags shift as libraries update, so check the Ollama library for current ones. For a web interface, pair Ollama with Open WebUI. For coding, add Continue or Tabby.
The 12GB Advantage: Two Models at Once
The real gain over 8GB is running two models side by side without swapping:
- Qwen3.5-9B Q4_K_M (5.68GB) + Qwen2.5-Coder 7B (~5GB) = ~10.7GB, chat and autocomplete together
- Parakeet (~2GB) + Step-Audio-EditX AWQ (~6-8GB) = ~8-10GB, a full speech pipeline
- Qwen3.5-9B Q4_K_M + mmproj (6.60GB) + Kokoro (<1GB) = ~7.5GB, see and speak with context to spare
Each total is static weights only. Leave a couple of gigabytes for KV cache on whichever model does the long-context work.
When to Upgrade
12GB handles most tasks. You would benefit from more if:
- You want Q8_0 quality on a 9B with real context room, rather than one or the other
- You want a 20B-class chat model such as GPT-OSS 20B, the 16GB pick
- You want Apriel-1.6-15B-Thinker with breathing room instead of a tight fit
- You want agentic coding, which starts to work at 16GB and is comfortable at 24GB
One caveat before you spend: 16GB does not fix vision. The open vision families jump from the 9B class straight to the 27-35B class, so 16GB lands between them and our deep dive leaves that tier’s vision pick explicitly unresolved. If vision quality is your reason for upgrading, the meaningful jump is to 24GB.
See our 16GB guide or 24GB guide for what each unlocks.
All eleven local-AI guides, by hardware tier and by task, are indexed at Local AI by VRAM: Which Models Fit Your GPU.