Running a model locally comes down to one number: how much memory your GPU has. Everything else, the quantization you pick, the context length you can afford, whether a model runs at all, follows from it. This page is the index to eleven guides covering five hardware tiers and six use cases, and it exists so you can find the one you need without reading the other ten.
Updated August 2026. Refreshed monthly as models and runtimes change.
Find Your Tier First
Check your card’s memory before anything else. On Windows, Task Manager’s Performance tab lists dedicated GPU memory. On Linux, nvidia-smi prints it. Then read across:
| Your GPU | Tier | Guide |
|---|---|---|
| RTX 4060, RTX 3070 | 8GB | 8GB VRAM |
| RTX 3060 12GB, RTX 4070 | 12GB | 12GB VRAM |
| RTX 4060 Ti 16GB, RTX 5060, Arc A770 | 16GB | 16GB VRAM |
| RTX 3090, RTX 4090 | 24GB | 24GB VRAM |
| RTX 5090 | 32GB | 32GB VRAM |
Each tier guide covers every task at that memory budget: chat, coding, vision, speech, translation and agents, with the specific model and quantization for each.
Or Start From the Task
If you already know your hardware and want depth on one job, the use-case guides compare models across all five tiers:
- Chat and general assistants - the everyday models, plus the open weights that are far too large for any consumer card and why that matters less than it sounds.
- Coding - autocomplete versus agentic coding, which are different jobs with different picks.
- Vision and OCR - image analysis and document understanding, including one tier where the honest answer is that there is no good pick.
- Speech - transcription and text to speech, where the licence often decides the recommendation rather than the benchmark.
- Translation - and the licensing traps, since several of the strongest translation models are non-commercial.
- Agents and tool use - function calling and multi-step workflows, the hardest thing to do well on local hardware.
How to Read the Numbers
Three things trip people up, and every guide here states them the same way.
A quoted size is the weights, not the total. When a guide says a model is 5.68GB at Q4_K_M, that is the weight file. The key-value cache grows with context length and sits on top, so a model whose weights fill your card will not run at a useful context. Anything within a gigabyte or two of your limit fits with minimal room to work in.
Quantization is the dial, and it is lossy. The same model exists at many precisions. Q4_K_M is the common default and roughly halves the size against Q8_0. Going below Q4 saves more memory and costs quality in ways that are rarely measured, which is why the guides stop recommending a model when the only way to fit it is a sub-Q4 build. Formats and their tradeoffs are documented in llama.cpp, which is the GGUF engine inside both Ollama and LM Studio, though both now ship an MLX engine alongside it on Apple Silicon.
Tag sizes move. Model libraries repack and requantize, so a size quoted anywhere goes stale. Read the current figure off the Ollama library tag list before you pull.
What These Guides Will Not Tell You
Worth being straight about the limits.
Almost every benchmark in this space is self-reported by the lab that trained the model. The comparisons on Qwen’s own model cards, for instance, are the vendor grading its own two product lines. That is still useful, and it is not independent verification. Each guide labels these where they appear rather than presenting them as measured results.
Some tiers have no good answer, and the guides say so. The 16GB vision tier is the clearest case: nothing fits without a lossy quantization, and no benchmark data exists at that precision for either candidate family. Writing “no pick” there is more useful than inventing one.
Apple Silicon does not fit this model at all. Macs use unified memory shared between CPU and GPU, so there is no discrete VRAM figure to look up, the usable fraction is governed by a system limit rather than the card, and memory bandwidth matters more than capacity. The tier tables here will mislead you on a Mac.
What This Means
The practical route is short. Find your memory figure, open the matching tier guide, and take the pick for the job you actually want to do. Use the task guides when you care about one job enough to want the comparison behind the recommendation, or when you are deciding what hardware to buy for a specific workload.
The models move faster than the hardware does. A card bought two years ago still sits in the same tier, but the best model for that tier has changed several times, which is why these pages are refreshed monthly rather than left to age.
The Bottom Line
VRAM sets the ceiling, quantization sets how close to it you can get, and context length quietly eats whatever is left. Everything in these eleven guides is downstream of those three facts. Start with your memory figure and work outward, and treat any single number you read anywhere, including here, as something to re-check against the model card before you commit an afternoon to a download.