Most local AI guides start at 8GB, which quietly excludes the GTX 1660, the RTX 2060, the RTX 3050 and a large share of gaming laptops sold in the last five years. Those cards run plenty. They just run different models, and the honest version of this page has more “no” in it than the higher tiers do.
How to read the VRAM numbers on this page. Every figure is the static size of the model weights at the named quantisation, taken from published GGUF file sizes or Ollama tag listings on 2026-08-06. The key-value cache is allocated on top and grows with context length, so treat each number as a floor. At 6GB the cache is proportionally a bigger problem than at 24GB, so leave more headroom than feels necessary.
Your Hardware
This tier covers the GTX 1660 and 1660 Ti, RTX 2060, RTX 3050, RTX 3060 laptop 6GB, and RTX 4050 laptop. If you are on a laptop, check whether the figure you are reading is dedicated VRAM or shared system memory, because the shared portion is far slower and will not save a model that does not fit.
Quick Reference
| Task | Model | Size | Quant | Notes |
|---|---|---|---|---|
| Chat | Qwen3.5-4B | 2.74 GB | Q4_K_M | Apache 2.0, room for real context |
| Chat (higher precision) | Qwen3.5-4B | 4.48 GB | Q8_0 | Tight, short context only |
| Coding | Qwen2.5-Coder 3B | 1.9 GB | Ollama build | FIM, leaves room for a chat model |
| Translation | TranslateGemma 4B | 3.3 GB | Ollama build | Custom Gemma terms, HF repo gated |
| Vision | Qwen3.5-4B + mmproj | 3.41 GB | Q4_K_M + mmproj-F16 | Keep image resolution modest |
| Speech (STT) | Parakeet TDT 0.6B v3 | ~2 GB | - | English plus 24 languages |
| Speech (TTS) | Kokoro-82M | <1 GB | - | Apache 2.0, CPU-capable |
| RAG | Qwen3-Embedding-0.6B + bge-reranker-v2-m3 | 1.08 GB | Q8_0 / Q4_K_M | Both Apache 2.0 |
| Agents | Qwen3.5-4B | 2.74 GB | Q4_K_M | Simple chains only, see below |
Chat & General Assistant
Qwen3.5-4B at Q4_K_M is 2.74GB, which is the pick. It is Apache 2.0 and ungated, and at under half your card it leaves a genuinely usable context window rather than the token-starved fit you get from forcing a bigger model.
The temptation at this tier is Qwen3.5-9B at Q4_K_M, 5.68GB. Resist it. That is 95% of a 6GB card before the KV cache exists, and it will either refuse to load or spill into system memory and run at a fraction of the speed. The 4B model answering quickly beats the 9B model swapping.
If you want more precision rather than more parameters, Qwen3.5-4B at Q8_0 is 4.48GB. That is the better upgrade path here, though it leaves little context room. Qwen3.5-2B at 1.28GB is the fallback when something else has to share the card.
Coding
Autocomplete: Qwen2.5-Coder 3B at 1.9GB. It is fill-in-the-middle trained, which the newer general-purpose models are not, and at under 2GB you can keep it resident alongside a chat model. The 7B build at 4.7GB also fits, but it takes almost the whole card and you lose the dual-model setup that makes local coding pleasant.
Agentic coding is not available at this tier. The models that do it are 27B and up. Use autocomplete locally and accept that multi-step repository work is a job for a bigger card.
Translation
TranslateGemma 4B at 3.3GB is purpose-built and beats a general model of the same size. Two caveats that apply at every tier: it ships under Google’s custom Gemma terms with a prohibited-use policy rather than a blanket open licence, and its Hugging Face repositories are gated behind a manual access request. The Ollama tags are not gated.
Vision
Qwen3.5-4B at Q4_K_M plus the 0.67GB mmproj-F16 projector is 3.41GB total, and every dense Qwen3.5 size is vision-capable, so one download covers chat and images. Keep image resolution modest: vision tokens consume context quickly, and context is the scarce resource here rather than weights.
Speech
Transcription: Parakeet TDT 0.6B v3 at roughly 2GB, covering English plus 24 European languages, CC-BY-4.0. Its small footprint is the whole argument at this tier, because it leaves room for a chat model to sit alongside it.
Text to speech: Kokoro-82M, under 1GB, Apache 2.0 and capable of running on CPU. Between them a full voice pipeline costs under 3GB, which is the one workload where a 6GB card is genuinely uncompromised.
RAG and Retrieval
This is the other place where 6GB is not a handicap. Qwen3-Embedding-0.6B is 639MB at Q8_0 and bge-reranker-v2-m3 is 438MB at Q4_K_M, about 1.08GB for the pair, both Apache 2.0. Retrieval quality on a 6GB card is identical to retrieval quality on a 5090, because these models are small everywhere. Our embedding and reranking guide covers the pairing in detail, including the fact that Ollama cannot rerank at all.
Agents (Tool Use)
Qwen3.5-4B handles single tool calls and two-step chains. The purpose-built agent models start at LFM2.5-8B-A1B, which is 5.16GB at Q4_K_M and does not leave enough room for the tool history an agent actually needs. Treat tool use here as a feature that works for simple automations rather than a workload the tier supports.
Getting Started
Install Ollama, then:
ollama pull qwen3.5:4b
ollama pull qwen2.5-coder:3b
ollama pull qwen3-embedding:0.6bThat is 5.9GB of downloads and roughly 3GB resident if you run one at a time, which is how you should run them at this tier.
Honest Limits
Worth being direct about what a 6GB card cannot do, because most guides skip this.
- No 8B or larger chat model at usable context. Qwen3.5-9B at Q4_K_M is 5.68GB and does not leave room to work in.
- No agentic coding. The capable models are five times your memory.
- No image generation beyond the small end. The smallest useful build in our image generation guide is Z-Image-Turbo at Q2_K, 3.64GB, and it still needs a text encoder on top. It is possible with encoder offloading, and it is not comfortable.
- Two models resident at once is usually out. Pick the dual setup only with the 3B coder plus a 2B chat model.
What a 6GB card does well is one job at a time, quickly. The voice pipeline and the retrieval stack are genuinely uncompromised, chat and vision at 4B are useful rather than a consolation, and the honest gap is agents and large-model reasoning. If those are what you need, the 12GB tier is where the picture changes, and the VRAM index covers every tier.