The release story for Qwen3.8-27B ran on 15 August: Apache-2.0 weights, an ungated repository, AMD numbers on day zero, and a benchmark table that wins some rows and loses others. That is all covered in the release piece, and the table stays there.
This is the question that comes next, and the one the announcements skip: what does a dense 27B model actually cost to hold on hardware someone already owns? The answer is mostly arithmetic, plus one tradeoff that almost never makes the release post.
The stock quants, in bytes
Local runs commonly start from a community GGUF conversion rather than the original weights. The file tree for unsloth/Qwen3.8-27B-GGUF, read through the Hugging Face API, reports an exact byte count for every file in the repository. Five of them span the range that matters for a single card:
| File | Bytes | On disk |
|---|---|---|
Qwen3.8-27B-Q8_0.gguf | 29,047,086,048 | 29.05 GB |
Qwen3.8-27B-Q4_K_M.gguf | 17,106,775,008 | 17.11 GB |
Qwen3.8-27B-IQ4_XS.gguf | 15,705,861,088 | 15.71 GB |
Qwen3.8-27B-UD-Q2_K_XL.gguf | 10,676,423,744 | 10.68 GB |
Qwen3.8-27B-UD-IQ2_XXS.gguf | 9,010,048,064 | 9.01 GB |
Against the Number of Parameters: 27B figure on the official model card, the Q8_0 file works out to a little over one byte per parameter. Q4_K_M is about 59 percent of the Q8_0 size, IQ4_XS shaves another 1.4 GB off that, and the two unsloth dynamic quants at the bottom of the table come in under 11 GB. Those are download sizes and disk sizes. They are the starting point of the memory question, not the answer to it.
The licensing and repository questions are settled elsewhere: the release piece covers the Apache-2.0 terms and the ungated repository, both checked against the API metadata. What follows is only the hardware question.
A file size is not a VRAM requirement
This is the most common way a local deployment plan goes wrong. The number in the tree describes the weights as stored on disk. A running process needs those weights, plus a KV cache that grows with every token held in context, plus the runtime’s own working allocations, plus whatever the image and video path needs when multimodal input is in play. The card calls the model “a native vision-language model that understands images and videos, with flexible thinking control”, so that last item is not hypothetical for anyone using it as intended.
How much extra sits on top of the file size depends on how much context is actually loaded and how the runtime handles its cache. The architecture sets the shape of the overhead rather than a single figure: config.json lists 64 layers, a hidden size of 5120, and a padded vocabulary of 248,320. Per-token cache cost follows from that architecture and the runtime’s cache settings. It cannot be read off a GGUF byte count. Anyone publishing one flat “VRAM required” number for a quant has quietly picked a context length and not told you which one.
The practical version is subtraction, and it has to be done in the units the board actually reports. A card sold as 16 GB holds 16 GiB, which is 17.18 GB in the decimal units Hugging Face uses for file sizes, and a 24 GB card holds 25.77 GB. IQ4_XS at 15.71 GB therefore leaves about 1.47 GB on a 16 GB board before a single cache block is allocated, and about 10.1 GB on a 24 GB one. Q4_K_M at 17.11 GB clears the 16 GB ceiling by roughly 73 MB, which is a fit on paper rather than one in practice once the runtime and any cache are in the picture, and leaves about 8.7 GB on a 24 GB board. Q8_0 at 29.05 GB is past 24 GB on weights alone, before context enters the discussion at all, so it calls for more memory than a 24 GB board holds: a larger card, several devices, or offload into system RAM. The distance between those situations is the entire decision.
262,144 tokens is a ceiling, not an allowance
The model card states the context length as “262,144 natively and extensible up to 1,000,000 tokens”, and config.json corroborates the native figure with text_config.max_position_embeddings = 262144. That is the number that goes in the headline.
The number that governs a fixed card is a different one: how much of that window can stay resident once the weights have taken their share. Because the cache grows with tokens held, the usable context is a function of what is left after the quant choice, which means quality and context are drawn from the same pool. Moving from IQ4_XS up to Q4_K_M buys back some fidelity and spends roughly 1.4 GB that the cache was going to use. Moving down to a two-bit dynamic quant does the reverse. There is no setting that gives both.
Thinking mode makes the arithmetic sharper. The card notes it is on by default and can be disabled per request, that reasoning depth is tunable through reasoning_effort, and that reasoning context from historical messages is retained via preserve_thinking. Retained reasoning is retained tokens, and retained tokens are cache. Images and video draw on the same window. A long conversation with thinking preserved and a few images in it will occupy far more of that 262,144 than the visible text suggests.
One rig, one shape
There is at least one detailed public account of pushing this model to its full context with the weights and KV cache on a single 24 GB card, the vision projector offloaded to a second GPU. An independent write-up dated 17 August 2026 describes a build targeting “a dense 27B model, its full 262,144-token context, multimodal input, maximum useful quality, and speculative decoding on an NVIDIA RTX PRO 4000 Blackwell SFF with 24 GB of VRAM”. Its author reports the finished system averaging 50.44 tok/s “in the current ten-run production series”, and states that “[a]t the far end of a genuinely occupied 256K cache, it still produces 12.61 tok/s without an out-of-memory failure”.
Those figures need a warning label. That system runs a custom llama.cpp build with a hand-tuned NVFP4/MTP quantisation and speculative decoding. It is not any of the stock GGUF files in the table above, and the numbers are one person’s machine, one build, and one workload. They are not out-of-the-box behaviour and should not be planned against as though they were.
The author is explicit that the two figures come from different test gates and “should stay separate”, warning that combining them “would make a better headline and a worse benchmark”, so the ratio between them is not something to quote. The qualitative point still carries, and it is the part to plan for: a long context is not a one-off cost paid at load time. It is a running cost paid on every token generated, and it gets heavier as the session fills.
What to check before the download starts
- The exact byte size of the specific file, not the family name. IQ4_XS and Q4_K_M differ by 1.4 GB, and that difference is cache headroom.
- What is left over after the weights on the target card, with room to spare rather than to the last hundred megabytes.
- How the runtime handles its KV cache, since cache precision settings change the leftover arithmetic.
- Whether the workload genuinely needs 262,144 tokens, or whether a smaller resident window buys back a larger quant.
- Whether thinking mode stays on and whether reasoning context is preserved across turns, because both land in the same cache.
The bottom line
The weights are the easy part. The hard part is that the two headline numbers people quote at each other, a quant’s file size and a 262,144-token context length, cannot both be spent at once on a fixed card.
An honest answer to “can this run on my GPU” is two numbers rather than one: how many bytes the file takes, and how much room is left above it. The second number is the one that decides how much of that context window is real. For a wider view of how the formats differ before picking one, the quantisation format comparison covers the same ground across GGUF, AWQ, GPTQ and MLX.