Local AI by VRAM Tier - 8GB | 12GB | 16GB | 24GB | 32GB
Deep dives: Chat | Coding | Translation | Vision | Speech | Agents
Drop a screenshot, a photo of a whiteboard, a scanned invoice, or a chart into a local vision-language model and get back a description, extracted text, or an answer about what is in the image. No upload, no per-image API charge, no copy of the document sitting on someone else’s disk.
Updated August 2026. Refreshed monthly as models and runtimes change.
What Changed in This Update
Earlier versions of this page recommended the Qwen3-VL family at every tier. That recommendation has been withdrawn at three of the five tiers. Qwen3.5 is a newer generation that supersedes both the Qwen3 text line and the Qwen3-VL vision line, its runtime support is now confirmed in llama.cpp, vLLM and Ollama, and vision is built into every dense size rather than split off into a separate vision line.
The per-tier result:
- 8GB: Qwen3-VL-8B out, Qwen3.5-9B in, with an important caveat about what the benchmark evidence does and does not cover.
- 12GB: the same Qwen3.5-9B, run at Q8_0 instead of Q4_K_M.
- 16GB: no recommendation. The tier is unresolved, and the reasoning is written out in full below rather than papered over with a pick.
- 24GB: Qwen3-VL-32B out, replaced by a choice between Qwen3.5-27B and the Qwen3.5-35B-A3B mixture-of-experts model.
- 32GB: the 24GB candidates with room to spare.
Two benchmark names have also been dropped. Earlier revisions of this page listed DocVQA and ChartQA as headline metrics for Qwen3-VL. Neither name appears in either family’s own reporting; Alibaba reports OmniDocBench1.5, CC-OCR, CharXiv (RQ) and AI2D_TEST in their place. Whatever the source of those older figures, it was not the Qwen model cards, so they are not carried forward.
How to Read the Numbers on This Page
Three caveats apply to everything below, and none of them are small.
1. Every benchmark figure here is vendor self-reported. All of the comparison numbers come from Alibaba’s own model cards, comparing Alibaba’s own two product lines against each other. No independent verification was found in this pass: OpenCompass’s OpenVLM leaderboard is a JavaScript application that returned no data to automated retrieval, and Artificial Analysis carries no vision benchmark for either family. Treat the tables as a vendor’s claim about its own products, not as a measured result.
2. There is no official Alibaba GGUF for Qwen3.5. Official GGUF builds were found for Qwen3-VL but not for Qwen3.5. Every usable Qwen3.5 GGUF available today is a third-party conversion, chiefly unsloth’s. That is a genuine asymmetry in favour of the older family: if a first-party quantisation matters to you, Qwen3-VL still has one and Qwen3.5 does not. The weight sizes quoted below are measured from those third-party files.
3. One vision bug is open against the MoE model. llama.cpp issue 25717, opened 2026-07-15 and still open, reports a CUDA illegal memory access on vision requests with Qwen3.5-35B-A3B plus an mmproj file, involving flash-attention. It bears directly on the 24GB mixture-of-experts option. A related vLLM issue, 34948, reports a CUDA illegal memory access in the GDN kernel and is also open.
How VRAM is counted here
Every VRAM figure on this page is a static weight size only and excludes the KV cache, which grows with context length and with image resolution. Each figure names its quantisation, and the vision projector (mmproj, shipped as a separate F16 file) is counted separately and then added in, because a vision model will not do vision without it. Anything landing within 1 to 2GB of a tier boundary fits with minimal context room, and that is flagged where it happens.
Qwen3-VL and Qwen3.5 Are Different Generations
This is the fact the older version of this page got wrong, so it is worth setting out plainly.
- The release timeline. The most recently touched Qwen3-VL repositories, the 30B-A3B and 235B-A22B builds, were last modified 2025-11-26, and the 8B and 32B repositories have not been touched since October 2025. The first Qwen3.5 repository was created 2026-02-16. No Qwen3-VL release has landed since Qwen3.5 shipped.
- Alibaba’s own framing. The Qwen3.5-9B card states, verbatim: “Unified Vision-Language Foundation: Early fusion training on multimodal tokens achieves cross-generational parity with Qwen3 and outperforms Qwen3-VL models across reasoning, coding, agents, and visual understanding benchmarks.” That is the vendor describing its own new line as a replacement for its own old one.
- Vision at every size. Qwen3.5-0.8B, 2B, 4B, 9B and 27B are all tagged
image-text-to-texton Hugging Face. The older arrangement split a text line from a separate-VLvision line; in Qwen3.5 there is no separate vision line to choose, and even the 0.8B model takes images. - Licensing. Both families are Apache-2.0 and ungated on Hugging Face.
Runtime support is the part that used to block this
Runtime support was the open question that kept earlier revisions on Qwen3-VL. It is now settled:
- llama.cpp: PR 19468 merged 2026-02-10. The PR body reads: “This PR adds model support for the upcoming Qwen3.5 models, including both dense and MoE variants. It has been verified with preview checkpoints from the Qwen Team in both vision and pure text modes (with or without mmproj file).”
- vLLM: PR 34110 merged 2026-02-09.
- Ollama: the official qwen3.5 library page carries tags from 0.8b through 122b, with capability tags reading “Text, Image”.
Anything downstream of llama.cpp, including LM Studio and Ollama’s llama.cpp path, needs a build newer than the February merge to run these weights in vision mode.
The Benchmarks These Models Are Scored On
Alibaba reports both families on the same set, which at least makes the two lines comparable to each other:
- MMMU and MMMU-Pro - college-level multi-discipline questions requiring image understanding
- MathVision and MathVista (mini) - mathematical reasoning from diagrams, equations and geometry
- RealWorldQA - questions about ordinary photographs
- MMStar - general multimodal reasoning
- OCRBench - text extraction accuracy from images
- OmniDocBench1.5 and CC-OCR - document and document-OCR benchmarks, used in place of DocVQA
- CharXiv (RQ) and AI2D_TEST - chart and diagram reasoning, used in place of ChartQA
- Pointing and grounding - can the model say where an object is, not just that it exists? Not part of Alibaba’s reported set; this is where Molmo2 specialises
Self-reported: the small Qwen3.5 models against Qwen3-VL-30B-A3B
Alibaba benchmarks its small dense models against the older family’s 30B mixture-of-experts model, not against the older 8B. All figures vendor self-reported.
| Benchmark | Qwen3.5-9B | Qwen3.5-4B | Qwen3-VL-30B-A3B |
|---|---|---|---|
| MMMU | 78.4 | 77.6 | 76.0 |
| MMMU-Pro | 70.1 | 66.3 | 63.0 |
| MathVision | 78.9 | 74.6 | 65.7 |
| MathVista (mini) | 85.7 | 85.1 | 81.9 |
| RealWorldQA | 80.3 | 79.5 | 77.4 |
| MMStar | 79.7 | 78.3 | 75.5 |
| OCRBench | 89.2 | 85.0 | 83.9 |
| OmniDocBench1.5 | 87.7 | 86.2 | 86.8 |
| CC-OCR | 79.3 | 76.7 | 77.8 |
Note the two places the 4B loses: OmniDocBench1.5 (86.2 against 86.8) and CC-OCR (76.7 against 77.8), both document-OCR tests.
Self-reported: Qwen3.5-27B against Qwen3-VL-235B-A22B
| Benchmark | Qwen3.5-27B | Qwen3-VL-235B-A22B |
|---|---|---|
| MMMU | 82.3 | 80.6 |
| MMMU-Pro | 75.0 | 69.3 |
| MathVision | 86.0 | 74.6 |
| MathVista (mini) | 87.8 | 85.8 |
| RealWorldQA | 83.7 | 81.3 |
| MMStar | 81.0 | 78.7 |
| OCRBench | 89.4 | 87.5 |
| OmniDocBench1.5 | 88.9 | 84.5 |
| CC-OCR | 81.0 | 81.5 |
Qwen3.5-27B loses one metric in Alibaba’s own table: CC-OCR, 81.0 against 81.5. It is a narrow loss on a single OCR benchmark against a model roughly nine times its total size, but it is the one place the vendor’s own numbers do not favour the new generation, and anyone whose workload is dominated by dense OCR should know it exists.
Weight Sizes by Quantisation
Static weight sizes measured from published GGUF files, in GB, excluding KV cache. The mmproj column is the F16 vision projector, required for image input.
| Model | Q4_K_M | Q8_0 | mmproj F16 | Q4_K_M + mmproj |
|---|---|---|---|---|
| Qwen3.5-2B | 1.28 | 2.01 | 0.67 | 1.95 |
| Qwen3-VL-4B | 2.50 | 4.28 | 0.84 | 3.34 |
| Qwen3.5-4B | 2.74 | 4.48 | 0.67 | 3.41 |
| Qwen3-VL-8B | 5.03 | 8.71 | 1.16 | 6.19 |
| Qwen3.5-9B | 5.68 | 9.53 | 0.92 | 6.60 |
| Qwen3.5-27B | 16.74 | 28.60 | 0.93 | 17.67 |
| Qwen3-VL-32B | 19.76 | 34.82 | 1.20 | 20.96 |
| Qwen3-VL-30B-A3B | 18.56 | 32.48 | 1.08 | 19.64 |
| Qwen3.5-35B-A3B | 22.02 | 36.90 | 0.90 | 22.92 |
Two lower-precision variants matter for the 16GB and 24GB tiers and sit outside the table: Qwen3.5-27B at IQ4_XS is 14.98GB and at Q6_K is 22.45GB, and Qwen3-VL-30B-A3B at Q3_K_M is 14.71GB.
8GB VRAM {#8gb}
GPUs: RTX 4060, RTX 3060 8GB, RTX 3070
Pick: Qwen3.5-9B (Q4_K_M + mmproj, 6.60GB)
Qwen3.5-9B at Q4_K_M is 5.68GB of weights plus a 0.92GB vision projector, 6.60GB in total. On an 8GB card that fits with minimal context room: budget carefully, keep image resolution modest, and expect the KV cache to consume most of what is left. The model it replaces, Qwen3-VL-8B at Q4_K_M plus mmproj, was 6.19GB, so the swap costs about 0.4GB.
The caveat that matters here. Alibaba did not benchmark Qwen3.5-9B against Qwen3-VL-8B. It benchmarked it against Qwen3-VL-30B-A3B, a much larger model from the older family, and won on all nine reported metrics. The recommendation therefore rests on an inference: a model that beats the older family’s 30B should also beat the older family’s 8B. That is a reasonable reading of the vendor’s own table, but it is not a direct same-size comparison, and no such comparison was found. Confidence is medium-high, not high. If your workload is narrow and Qwen3-VL-8B is already tuned into it, there is no measured result forcing a change.
Smaller fallbacks. Qwen3.5-4B at Q4_K_M plus mmproj is 3.41GB and Qwen3.5-2B is 1.95GB, both vision-capable, and a 0.8B model exists in the same line. The 4B beats Qwen3-VL-30B-A3B on seven of the nine reported metrics but loses both document-OCR tests, so prefer the 9B for receipts, invoices and scans.
Alternatives from other families. InternVL3.5-8B remains a reasonable second opinion on multi-step visual reasoning; run it 8-bit to fit a consumer card. MiniCPM-V 4.6, at 1.3B parameters and 2.60GB of BF16 weights, or 1.52GB as an F16 GGUF plus a 1.11GB projector, is still the most capability per gigabyte at the very low end, and the option that lets vision run alongside a chat model on a small card.
For all use cases at this level, see the 8GB VRAM complete guide.
12GB VRAM {#12gb}
GPUs: RTX 3060 12GB, RTX 4070
Pick: Qwen3.5-9B at Q8_0 (10.45GB with mmproj)
Same model as the 8GB tier, run at a much higher precision instead of a larger parameter count. Q8_0 weights are 9.53GB, plus the 0.92GB projector, for 10.45GB. On a 12GB card that again fits with minimal context room, so it is a real trade: cleaner output on small text and fine detail, less room for long context and high-resolution images. Dropping back to Q4_K_M at 6.60GB buys roughly 3.9GB of headroom whenever a job needs it.
Grounding specialist. Molmo2-8B is unchanged as the open option for pointing at UI elements, locating objects and tracking across frames rather than general question answering. No firm VRAM figure was found in Ai2’s announcement, so budget conservatively and test on your own card.
For all use cases at this level, see the 12GB VRAM complete guide.
16GB VRAM {#16gb}
GPUs: RTX 4060 Ti 16GB, RTX 5060, Intel Arc A770, AMD RX 7800 XT
Unresolved: this page makes no recommendation at this tier
This is the awkward tier, and rather than manufacture a pick to make the page look complete, here is the actual state of the evidence.
The interesting question at 16GB is whether to squeeze in a model from the 27B-to-30B class. Neither family fits there without a lossy quantisation:
| Option | Quantisation | Weights + mmproj | Free on a 16GB card |
|---|---|---|---|
| Qwen3.5-27B | IQ4_XS | 14.98 + 0.93 = 15.91GB | almost nothing |
| Qwen3-VL-30B-A3B | Q3_K_M | 14.71 + 1.08 = 15.79GB | almost nothing |
Three things block a verdict:
- Both options require quantisation levels below Q4_K_M, where quality loss is real but unmeasured for these specific models. Qwen3.5-27B needs Q3_K_M or IQ4_XS to fit at all; Qwen3-VL-30B-A3B needs Q3_K_M.
- No benchmark data exists at those quantisation levels for either family. Every number in the tables above is reported at full precision. Carrying a full-precision win down to a 3-bit or IQ4 build is an assumption, not a measurement, and it is exactly the kind of assumption that fails asymmetrically between a dense model and a mixture-of-experts model.
- The MoE speed advantage is real but unquantified. Qwen3-VL-30B-A3B activates roughly 3B parameters per token, so it should generate faster than a dense 27B on the same card. No throughput comparison at these quantisation levels was found, so “faster” cannot be turned into a number, and it cannot be traded off against a quality difference that is also unmeasured.
Both configurations also leave under a gigabyte free before the KV cache, which for a vision model processing high-resolution images is not a workable margin on its own.
A reader who needs something working today can run the 12GB configuration here, Qwen3.5-9B at Q8_0 and 10.45GB, with about 5.5GB of genuine headroom for context, a second model, or both. That is a smaller model comfortably run, not an answer to the 16GB question, and it should not be read as one.
For all use cases at this level, see the 16GB VRAM complete guide.
24GB VRAM {#24gb}
GPUs: RTX 3090, RTX 4090
Two Qwen3.5 candidates fit, and they answer different questions. Both replace the previous Qwen3-VL-32B recommendation.
| Option | Quantisation | Weights + mmproj | Character |
|---|---|---|---|
| Qwen3.5-27B | Q6_K | 22.45 + 0.93 = 23.38GB | quality pick, dense |
| Qwen3.5-35B-A3B | Q4_K_M | 22.02 + 0.90 = 22.92GB | speed pick, mixture-of-experts |
Quality pick: Qwen3.5-27B at Q6_K
Q6_K keeps quantisation loss low on the model that carries the strongest self-reported vision numbers in the family. At 23.38GB with the projector it fits a 24GB card with minimal context room, which on a dense model with high-resolution image input is tight. Q4_K_M plus mmproj at 17.67GB is the fallback that buys back nearly 6GB.
Speed pick: Qwen3.5-35B-A3B at Q4_K_M
The mixture-of-experts model lands at a similar 22.92GB but activates only about 3B parameters per token, which is the standard reason to prefer an MoE for throughput-heavy work such as batch OCR. Two things to weigh against that:
- The open bug applies to this model specifically. llama.cpp issue 25717, opened 2026-07-15 and still open, reports a CUDA illegal memory access on vision requests with Qwen3.5-35B-A3B plus mmproj, involving flash-attention. Anyone choosing this option should check the issue’s current state and test their own flash-attention configuration before building a pipeline on it.
- Higher-precision builds exist but overflow the tier. Q6_K is 28.85GB and Q8_0 is 36.90GB, so on a 24GB card Q4_K_M is the only realistic option and the dense 27B keeps more usable precision headroom.
The newer option at this tier: Qwen3.6
Qwen3.6 was created in April 2026, is Apache-2.0 and vision-capable, and ships in only two sizes, 27B and 35B-A3B. Alibaba’s self-reported figures put Qwen3.6-27B slightly ahead of Qwen3.5-27B: MMMU 82.9 against 82.3, RealWorldQA 84.1 against 83.7, and OCRBench tied at 89.4. That is a small margin from the same self-reporting source as everything else here.
The tiers on this page are not restructured around it for one reason: there is no 4B or 9B Qwen3.6, so it cannot fill the 8GB or 12GB brackets at all. Ollama now publishes Qwen3.6 builds, with the dense 27B at 17GB and the 35B-A3B at 24GB, which places the dense model almost exactly where Qwen3.5-27B lands. Those figures include Ollama’s bundled projector rather than being weights-only, so they are not directly comparable with the table above. At 24GB and above it is worth testing against the Qwen3.5 picks.
For all use cases at this level, see the 24GB VRAM complete guide.
32GB VRAM {#32gb}
GPUs: RTX 5090
Pick: the 24GB candidates, with room to actually use them
Nothing new becomes available at 32GB; what changes is that the 24GB options stop being tight. Qwen3.5-27B at Q6_K plus mmproj, 23.38GB, leaves roughly 8.6GB for context and high-resolution images instead of roughly 0.6GB, which is the difference between a demo and a working pipeline. Qwen3.5-35B-A3B at Q4_K_M plus mmproj, 22.92GB, gets the same benefit.
Going further, Qwen3.5-27B at Q8_0 is 28.60GB of weights plus the 0.93GB projector, 29.53GB, which fits a 32GB card with minimal context room. No benchmark comparison between Q6_K and Q8_0 builds of this model was found, so treat the higher precision as a hedge against quantisation artifacts on dense small text rather than as a measured gain.
Multi-model stack. 32GB is also enough to keep vision, chat and speech resident at once: Qwen3.5-9B at Q4_K_M plus mmproj (6.60GB) for images, a 14B-class chat model, and a speech pair such as Parakeet and Kokoro, with margin left over. Point a camera at something, ask about it, and hear the answer, all local.
For all use cases at this level, see the 32GB VRAM complete guide.
Cross-Tier Summary
Weights plus vision projector, excluding KV cache.
| Tier | Pick | Quantisation | VRAM | Note |
|---|---|---|---|---|
| 8GB | Qwen3.5-9B | Q4_K_M | 6.60 GB | Fits with minimal context room; rests on an inference, not a same-size test |
| 12GB | Qwen3.5-9B | Q8_0 | 10.45 GB | Fits with minimal context room; drop to Q4_K_M for headroom |
| 16GB | No pick | - | - | Unresolved: both 27B/30B options need sub-Q4 quants with no benchmark data |
| 24GB | Qwen3.5-27B | Q6_K | 23.38 GB | Quality pick; minimal context room |
| 24GB | Qwen3.5-35B-A3B | Q4_K_M | 22.92 GB | Speed pick; open llama.cpp vision bug 25717 |
| 32GB | Qwen3.5-27B | Q6_K or Q8_0 | 23.38 or 29.53 GB | Same models, usable context budget |
What Vision Models Actually Do Well
The practical use cases that hold up today:
- OCR and document reading - text out of photos, scanned PDFs, receipts and invoices. This is the area both Qwen families are built around, and the area where the one benchmark loss noted above sits
- Screenshot analysis - describe what is on screen, identify UI elements, read error messages
- Chart and diagram reading - extract data points, describe trends, answer questions about a figure
- Pointing and grounding - locate a specific object or UI element by coordinates, which is Molmo2’s specialty
- Whiteboard to text - photograph handwritten notes and get structured text back
- Math from images - photograph an equation or geometry problem and get it worked through
What still goes wrong:
- Spatial reasoning - “is the red object to the left of the blue one?” is more reliable than a year ago but still trips smaller models
- Counting - “how many people are in this photo?” fails often on crowded scenes
- Fine-grained detail - very small text and subtle differences between similar objects, which is the main argument for spending VRAM on precision rather than parameters
- Long video understanding - improving, but still slow and memory-hungry at length
Quick Start
# Pull a vision model. Confirm the exact tag on the model page first;
# the qwen3.5 library lists sizes from 0.8b to 122b.
ollama pull qwen3.5:9b # 8GB and 12GB tiers
ollama pull qwen3.5:27b # 24GB+ tier
# Analyse an image from the command line
ollama run qwen3.5:9b "Describe this image" ./photo.jpg
# Or use the API
curl http://localhost:11434/api/chat -d '{
"model": "qwen3.5:9b",
"messages": [{
"role": "user",
"content": "What text is in this image?",
"images": ["base64_encoded_image_here"]
}]
}'Vision support depends on the runtime build, not just the model file. llama.cpp gained Qwen3.5 support, vision included, in a February 2026 merge, so anything built on an older llama.cpp cannot load these weights in vision mode. The mmproj projector file has to be present as well; without it the model loads as text-only.
For a browser interface with image upload, pair Ollama with Open WebUI; the setup is covered in the Ollama and Open WebUI guide.
All eleven local-AI guides, by hardware tier and by task, are indexed at Local AI by VRAM: Which Models Fit Your GPU.