Local AI by VRAM Tier - 8GB | 12GB | 16GB | 24GB | 32GB
Deep dives: Chat | Coding | Translation | Vision | Speech | Agents
12GB is where local AI goes from “usable” to “actually good.” You can run 14B models with context headroom, fit two smaller models simultaneously, and run speech pipelines that combine transcription with voice synthesis. It’s the sweet spot between budget and capability.
Your Hardware
12GB VRAM cards: RTX 3060 12GB ($200-250 used), RTX 4070 ($500 new). The RTX 3060 12GB is arguably the best value GPU for local AI - 12GB at a fraction of the price of higher-end cards.
The math: At Q4_K_M, 12GB fits models up to about 14B parameters with 1-2GB for context. At Q3_K_M (more aggressive compression), you can squeeze in 20B models but with quality loss. The comfortable zone is 8-14B at Q4.
Quick Reference
| Use Case | Best Pick | VRAM Used | Key Score | Speed (RTX 4070) |
|---|---|---|---|---|
| Chat | Qwen 3 14B | 10.7 GB | 81.1 MMLU | ~50 tok/s |
| Coding (autocomplete) | Qwen 2.5 Coder 14B | ~9 GB | ~89% HumanEval | ~45 tok/s |
| Coding (chat) | Qwen 3.5 9B | ~7 GB | 65.6% LiveCodeBench | ~55 tok/s |
| Translation | TranslateGemma 12B | 8.1 GB | 83.5 COMET22 | Fast |
| Vision | Qwen3-VL 8B | ~6 GB | 85.8 MathVista | 80-120 tok/s |
| Speech (STT) | Whisper large-v3-turbo | ~6 GB | ~7.75% WER | 6x real-time |
| Speech (TTS) | Chatterbox Turbo | ~4 GB | Beats ElevenLabs | Sub-200ms |
| Agents | Qwen 3 14B | 10.7 GB | ~62 BFCL V4 | ~50 tok/s |
Chat & General Assistant
The jump from 8B to 14B is where local models start feeling smart. Qwen 3 14B at 10.7GB scores 81.1 MMLU and 92.5 GSM8K. Longer, more coherent responses. Better follow-up handling. Noticeably more accurate facts.
Reasoning specialist: DeepSeek-R1 14B Distill at ~9GB runs at 45 tok/s. Chain-of-thought traces make it ideal for debugging logic and working through math.
Full comparison: local chat model guide.
Coding
Autocomplete: Qwen 2.5 Coder 14B at ~9GB with ~89% HumanEval, FIM support, and 128K context. The clear upgrade from the 7B - fewer hallucinated APIs, better framework understanding.
Alternative: Codestral 22B at Q4 (~11GB). Strong multi-language support (73.75% Kotlin-HumanEval) for polyglot developers.
Full comparison: local coding model guide.
Translation
TranslateGemma 12B at 8.1GB is the inflection point. At MetricX 3.60 and COMET22 83.5, it beats the Gemma 3 27B baseline while using half the VRAM. Near-commercial quality for 55 languages.
Full comparison: local translation model guide.
Vision
Qwen3-VL 8B runs with 6GB headroom at this tier, delivering 80-120 tok/s. Or try Phi-4-reasoning-vision 15B at ~10GB for STEM-specific visual reasoning (75.2 MathVista, 83.3 ChartQA).
Full comparison: local vision model guide.
Speech
12GB is where simultaneous STT + TTS becomes possible. Whisper large-v3-turbo (6GB) + Chatterbox Turbo (4GB) = ~10GB total. Commercial-grade transcription and voice cloning running side by side with 2GB headroom.
Chatterbox beat ElevenLabs in blind evaluations (63.75% preference rate) with sub-200ms latency.
Full comparison: local speech model guide.
Agents
Qwen 3 14B with native tool calling handles 4-5 step workflows reliably. More consistent structured output than the 8B models. At 10.7GB, there’s limited context headroom - if your agent processes large tool responses, consider Qwen 3.5 9B at 7GB for the extra buffer.
Full comparison: local agent model guide.
Getting Started
# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh
# Pull essentials for 12GB
ollama pull qwen3:14b # Chat + agents
ollama pull qwen2.5-coder:14b # Coding autocomplete
ollama pull translategemma:12b # Translation
# Start chatting
ollama run qwen3:14bThe 12GB Advantage: Dual Models
The real power of 12GB is running two models simultaneously:
- Coder 7B (5GB) + Chat 3.5 4B (3.4GB) = 8.4GB - autocomplete + quick chat
- Whisper turbo (6GB) + Chatterbox Turbo (4GB) = 10GB - full speech pipeline
- Vision 8B (6GB) + TTS Kokoro (<1GB) = ~7GB - see and speak
This flexibility is what makes 12GB feel like a generation ahead of 8GB.
When to Upgrade
12GB handles most use cases well. You’d benefit from 16GB if:
- You want 14B models at Q5 quantization (better quality)
- You need long context with 14B models (more headroom)
- You want to run Gemma 3 27B QAT for vision (needs 14GB)
See our 16GB guide for what the next tier unlocks.