Local AI by VRAM Tier - 8GB | 12GB | 16GB | 24GB | 32GB
Deep dives: Chat | Coding | Translation | Vision | Speech | Agents
August 2026 refresh: the picks here mostly hold. Vision moves to Qwen3.5-9B, so one download covers chat and images, and whisper.cpp now runs Parakeet natively. Every VRAM figure below is a static weight size with the quantisation named.
You have 8GB of VRAM. A year ago that meant toy models and disappointment. In 2026 it means a genuinely useful local AI setup across chat, coding, translation, vision, speech, and basic agent tasks, all running privately on your own hardware.
Updated August 2026. Refreshed monthly as models and runtimes change.
Your Hardware
8GB VRAM cards include the RTX 4060, RTX 3060 8GB, RTX 3070, RTX 2080, and GTX 1080. Street prices move constantly, so check current listings.
How to read the VRAM numbers here. Each figure is the static weight size at the named quantisation, from published GGUF files; the KV cache is allocated on top, so treat every number as a floor. Anything within 1-2GB of the 8GB ceiling fits with minimal context room. As rough math, Q4_K_M needs roughly 0.6-0.8GB per billion parameters, so 8GB holds up to about the 9B class.
Quick Reference
| Use Case | Best Pick | Static weights | Quantisation | Note |
|---|---|---|---|---|
| Chat | Qwen3.5-9B | 5.68 GB | Q4_K_M | Apache-2.0, current Qwen generation |
| Coding (autocomplete) | Qwen2.5-Coder 7B | ~5 GB | Q4_K_M | 88.4% HumanEval, FIM |
| Coding (chat) | Qwen3.5-9B | 5.68 GB | Q4_K_M | Same download as chat |
| Translation | TranslateGemma 4B | 3.3 GB | Ollama build | Custom Gemma terms |
| Vision | Qwen3.5-9B + mmproj | 6.60 GB | Q4_K_M + mmproj-F16 | Minimal context room |
| Speech (STT) | Parakeet TDT 0.6B v3 | ~2 GB | - | English plus 24 languages |
| Speech (TTS) | Kokoro-82M | <1 GB | - | Apache-2.0, CPU-capable |
| Agents | LFM2.5-8B-A1B | 5.16 GB | Q4_K_M | Tool calling, LFM Open License v1.0 |
Chat & General Assistant
Qwen3.5-9B is the all-round 8GB chat model, Apache-2.0 and ungated. Static weights are 5.68GB at Q4_K_M, leaving a usable couple of gigabytes for the KV cache, and it handles everyday questions, brainstorming, summarisation and light analysis. Gemma 4 E4B is the alternative if you want a different model family, though it is the larger of the two rather than the smaller: Ollama’s default build is 9.6GB and the smallest published build, the QAT one, is 6.1GB.
Full comparison: local chat model guide.
Coding
Autocomplete: Qwen2.5-Coder 7B at 88.4% HumanEval with fill-in-the-middle and 128K context, per Table 16 of the Qwen team’s own technical report. At roughly 5GB it leaves room for context. Plug it into Continue or Tabby for inline suggestions. It is deliberately an older generation than the chat pick, because the newer general-purpose Qwen releases are not FIM-trained.
Chat-based coding: Qwen3.5-9B again, the same download as your chat model.
Full comparison and IDE setup: local coding model guide.
Translation
TranslateGemma 4B is listed on Ollama at 3.3GB, small enough to sit alongside another model, and a purpose-built translator beats a general model of the same size. One licensing point that gets glossed over: it ships under Google’s custom Gemma terms with a prohibited-use policy attached, not a blanket open licence. For very rare pairs, NLLB-200 covers 200 languages, though its 1.3B distilled build is published only as a 5.48GB FP32 checkpoint, so size it against your loader’s precision rather than its parameter count. The output is literal and the licence is CC-BY-NC-4.0, so non-commercial only.
Full comparison: local translation model guide.
Vision
The pick changes this month: Qwen3-VL-8B is out, Qwen3.5-9B is in. Qwen3.5 supersedes both the Qwen3 text line and the Qwen3-VL vision line, whose generative repositories have had no release since (last modified November 2025, against February 2026 for the first Qwen3.5 repository; only the Qwen3-VL embedding and reranker repositories have been touched since), and every dense Qwen3.5 size is vision-capable, so one download covers chat and images. At Q4_K_M the weights are 5.68GB plus a 0.92GB mmproj-F16 projector, 6.60GB in total: minimal context room on an 8GB card, so keep image resolution modest.
The caveat. Alibaba benchmarked Qwen3.5-9B against the much larger Qwen3-VL-30B-A3B, not against Qwen3-VL-8B, reporting MMMU 78.4 against 76.0 and OCRBench 89.2 against 83.9. Those figures are self-reported by Alibaba across its own two product lines, and no independent verification was found. The pick rests on an inference, that a model beating the older family’s 30B should also beat its 8B, not on a same-size test. Confidence is medium-high, not high.
Two practical notes. Alibaba publishes no official GGUF for Qwen3.5, only for Qwen3-VL, so every Qwen3.5 GGUF is third-party work such as the Unsloth builds. Runtime support is settled: llama.cpp PR #19468 merged in February 2026, verified in “both vision and pure text modes”, and the Ollama library marks qwen3.5 tags “Text, Image”.
Ultralight alternative: MiniCPM-V 4.6, a 1.3B model: 2.60GB of BF16 weights, or 1.52GB as an F16 GGUF plus a 1.11GB projector.
Full comparison: local vision model guide.
Speech
This is where 8GB changed most. Transcription: NVIDIA Parakeet TDT 0.6B v3 loads in about 2GB and covers English plus 24 more languages. Whisper large-v3-turbo at roughly 6GB is the pick for broad multilingual audio. No accuracy ranking between them is restated this month: the models are unchanged, but the live Open ASR leaderboard order could not be reliably reconfirmed in this pass.
The runtime update worth acting on. whisper.cpp is at v1.9.2 (4 August 2026), and v1.9.0 (17 June 2026) added native NVIDIA Parakeet support, so the English pick and the multilingual pick can share one runtime instead of pairing whisper.cpp with a separate NeMo install.
Text-to-speech: Kokoro-82M is Apache-2.0, runs at a fraction of a gigabyte, works on CPU if needed, and has built-in voices across 8 languages with no cloning.
Both together: at about 2GB for Parakeet and under 1GB for Kokoro, transcription, synthesis and a small chat model fit at once on one 8GB card, a full voice pipeline that used to need a 12GB or larger GPU.
Full comparison and pipeline setups: local speech model guide.
Agents (Tool Use)
LFM2.5-8B-A1B is an 8.3B-total / 1.5B-active mixture-of-experts model built for on-device agents: it writes Pythonic function calls by default, is 5.16GB at Q4_K_M in Liquid AI’s own GGUF, and posts strong tool-use scores (Tau2-Bench Telecom 88.07, BFCLv4 48.50, both self-reported). Check the licence before shipping: it is not Apache or MIT but Liquid AI’s own LFM Open License v1.0, which licenses commercial use only below $10 million in annual revenue. It handles 2-3 step tool chains reliably, enough for smart-home control, single API calls, and basic automations. Qwen3.5-9B is the alternative when one model for both chat and tools is the simpler answer.
Full comparison and setup: local agent model guide.
Getting Started
# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh
# Pull the essentials for 8GB
ollama pull qwen3.5:9b # Chat + coding chat + vision
ollama pull qwen2.5-coder:7b # Autocomplete
# Start chatting
ollama run qwen3.5:9bThe qwen3.5 tags carry “Text, Image” capabilities, so the same pull covers vision. For a web interface, pair Ollama with Open WebUI. For coding, add Continue or Tabby. Model tags shift as libraries update, so check the Ollama library for the current ones.
Honest Limits
8GB is the entry tier. What it still cannot do well:
- Complex multi-step reasoning - models under the 14B class lose coherence on long logic chains
- Two large models at once - fitting two 7-9B models simultaneously is not realistic; you switch between them, with small speech and vision models the exception noted above
- Large context - a 5.68GB chat model leaves roughly 2GB for the KV cache, and the 6.60GB vision build leaves closer to 1GB, so long documents get truncated
- Reliable long agents - 2-3 step chains work, 5-plus step workflows fail often
If you keep hitting these walls, a 12GB card (RTX 3060 12GB) or a 16GB card (RTX 4060 Ti 16GB) is a meaningful upgrade. See the 12GB guide or 16GB guide.
All eleven local-AI guides, by hardware tier and by task, are indexed at Local AI by VRAM: Which Models Fit Your GPU.