Open-weight model families now publish both styles side by side. Qwen ships Qwen3-32B (dense) and Qwen3-30B-A3B (MoE); Mistral’s Mixtral-8x7B is MoE; IBM ships Granite 4.0 in both dense sizes (Micro, H-Micro) and MoE sizes (H-Tiny, H-Small) (Granite-4.0-H-Small model card). The “A3B” or “A22B” suffix is the key: it tells you how many parameters are activated per token, which sets compute per token and so, roughly, generation speed. Memory is a different story: every expert still has to be stored somewhere, so the total parameter count sets how much VRAM plus system RAM the weights need.
What the suffix means, exactly
A dense model uses every parameter on every token. Qwen3-32B’s config.json lists no expert fields and an architectures entry of Qwen3ForCausalLM: each of its 64 layers runs its full MLP for every token (Qwen3-32B config). A Mixture-of-Experts model partitions the MLP into many smaller MLPs (“experts”) and routes each token to a small subset of them.
Qwen3-30B-A3B’s config carries num_experts: 128 and num_experts_per_tok: 8, with architectures: ["Qwen3MoeForCausalLM"] (Qwen3-30B-A3B config). The model card states “30.5B in total and 3.3B activated” (Qwen3-30B-A3B model card). Qwen3-235B-A22B-Instruct-2507 has the same expert layout (128 experts, 8 active per token) across 94 layers instead of 48, and its card states “235B in total and 22B activated” (Qwen3-235B-A22B-Instruct-2507 model card).
The classic third-party case is Mixtral-8x7B, where num_local_experts: 8 and num_experts_per_tok: 2 mean each token sees two of the eight experts (Mixtral config). Mistral’s announcement puts it at “46.7B total parameters but only uses 12.9B parameters per token” (Mistral AI). DeepSeek-V3.2 is the most aggressive current example: n_routed_experts: 256, n_shared_experts: 1, num_experts_per_tok: 8 (DeepSeek-V3.2 config). Its model card gives no parameter figures; the Hugging Face page header shows “685B params”. Its config matches DeepSeek-V3’s expert layout field for field (same hidden size, expert width, 256 routed experts, 8 per token, 61 layers; V3.2 adds only the sparse-attention indexer fields), and the DeepSeek-V3 card states “671B total parameters with 37B activated for each token” (DeepSeek-V3 model card). Roughly 37B activated for V3.2 is therefore an inference from the shared layout, not a figure DeepSeek publishes for V3.2.
The point: total parameters predict storage and memory footprint; activated parameters predict compute per token. Among the models below the ratio runs from about 3.6x (Mixtral, Granite H-Small) to about 18x (DeepSeek-V3’s 671B total vs 37B activated), which is the whole reason MoE is interesting locally.
A side-by-side comparison
Figures below come from each model’s Hugging Face API entry, config.json and card, fetched on 2026-09-30. The Hugging Face API’s safetensors.total field is a parameter count, not a byte count. “Weights” is computed from the per-type parameter counts in the same API response: BF16 is 2 bytes per parameter, FP8 is 1 byte. GB means 10^9 bytes and GiB means 2^30 bytes. These are the published full-precision files; a 4-bit download is roughly 30 percent of the BF16 size (Ollama lists qwen3:32b at 20GB against 65.52 GB in BF16).
| Model | Architecture | Total params | Active / token | Weights as published | Licence | HF gated |
|---|---|---|---|---|---|---|
| Llama-3.1-8B | Dense | 8.03B | 8.03B | 16.06 GB / 14.96 GiB (BF16) | llama3.1 | manual |
| Qwen3-32B | Dense | 32.76B | 32.76B | 65.52 GB / 61.02 GiB (BF16) | apache-2.0 | false |
| Mixtral-8x7B-Instruct-v0.1 | MoE (2 of 8) | 46.7B | 12.9B (per Mistral) | 93.41 GB / 86.99 GiB (BF16) | apache-2.0 | false |
| Qwen3-30B-A3B | MoE (8 of 128) | 30.5B | 3.3B | 61.06 GB / 56.87 GiB (BF16) | apache-2.0 | false |
| granite-4.0-h-small | Hybrid Mamba2 / attention MoE (10 of 72) | 32B (per IBM) | 9B (per IBM) | 64.41 GB / 59.99 GiB (BF16) | apache-2.0 | false |
| Qwen3-235B-A22B-Instruct-2507 | MoE (8 of 128) | 235B | 22B | 470.19 GB / 437.90 GiB (BF16) | apache-2.0 | false |
| DeepSeek-V3.2 | MoE (8 of 256 routed + 1 shared) | 685B (HF page) | ~37B (inferred from the V3 layout) | 689.30 GB / 641.97 GiB (mostly FP8) | MIT | false |
Sources: Llama-3.1-8B API, Qwen3-32B API, Mixtral API, Qwen3-30B-A3B API, granite-4.0-h-small API, Qwen3-235B-A22B API, DeepSeek-V3.2 API. DeepSeek-V3.2’s weight bytes: 681.41B FP8 parameters at 1 byte, 3.95B BF16 parameters at 2 bytes, plus about 1.1M FP32 parameters.
Notice the column that does not move with total parameters: active per token. Qwen3-30B-A3B and Qwen3-32B are close in size (61.06 GB vs 65.52 GB in BF16; Ollama lists the default 4-bit tags qwen3:30b-a3b at 19GB and qwen3:32b at 20GB on its tag page), but Qwen3-30B-A3B does roughly 3.3B-parameter math per token while Qwen3-32B does 32B-parameter math. So at the same quant the MoE model needs about the same memory but generates tokens faster.
MoE also changes what “fits”. Because only a few experts run per token, the expert weights can sit in system RAM while the rest of the model stays on the GPU. llama.cpp exposes this directly: --cpu-moe will “keep all Mixture of Experts (MoE) weights in the CPU”, and --n-cpu-moe N does the same for the first N layers (llama.cpp server README). The experts still have to be resident in RAM; streaming them from SSD is not a llama.cpp feature (the request to stream routed experts from SSD was closed as not planned, issue 25257, and a newer just-in-time streaming request, issue 27562, is open).
When MoE is the right pick locally
MoE wins on three workloads in particular.
A GPU smaller than the model, with plenty of system RAM. With a dense model, layers that do not fit in VRAM run on the CPU at full per-token cost. With an MoE model, the expert weights can be kept in system RAM (--cpu-moe / --n-cpu-moe above) while attention and shared weights stay on the GPU, and only the few selected experts are computed per token. The memory budget is still set by the total: VRAM plus RAM must hold every expert.
Generation speed. Per-token compute is set by the activated subset, not the total parameter count, so a 30B-total MoE with 3.3B active tends to generate faster than a 32B dense model at the same quant. Quality claims are a separate question: the Qwen3-235B-A22B-Instruct-2507 card reports its own (vendor-reported) benchmark results alongside DeepSeek-V3-0324, GPT-4o, Claude Opus 4 (non-thinking) and Kimi K2, while computing 22B parameters per token.
Cost crossover. The “active compute” number feeds the local-vs-API crossover through speed: more tokens per second on the same hardware means a lower cost per token locally. The hardware bill, though, is set by the total: Qwen3-235B-A22B-Instruct-2507 is 470.19 GB (437.90 GiB) in BF16, so even a 4-bit copy needs well over 100 GB of VRAM plus RAM.
When dense is still the right pick locally
Three cases where MoE does not help.
Small models. The small tiers ship dense: Qwen3’s open-weight dense line is 0.6B, 1.7B, 4B, 8B, 14B and 32B, with MoE only at 30B-A3B and 235B-A22B (Qwen blog). See the without-a-GPU guide for the small tier in detail.
Everything already fits in VRAM and memory is the limit. MoE saves compute, not memory: at 4-bit, qwen3:30b-a3b (19GB) and qwen3:32b (20GB) need about the same space (Ollama). If the goal is the most model that fits in a fixed VRAM budget, total parameters are what count.
Inference engines with open MoE bugs on your hardware. Ollama has open MoE crash reports on Blackwell (issue 18276), on mixed-architecture AMD ROCm systems (issue 18162) and for MLX Gemma 4 MoE imports (issue 18540). These are not deal-breakers for the listed models, but they are worth checking against the engine and hardware you actually run.
How to read a model card in 30 seconds
A four-field check tells you almost everything.
safetensorson the HF API page: the parameter count per data type. Multiply by bytes per parameter (2 for BF16, 1 for FP8) for the weight size at the published precision.model_typeandarchitecturesinconfig.json: names containingmoeormixtral(for exampleQwen3MoeForCausalLM,GraniteMoeHybridForCausalLM) are MoE. Not every MoE says so in its name (DeepseekV32ForCausalLMdoes not), so check field 3.num_experts(ornum_local_experts,n_routed_experts) andnum_experts_per_tok: if present, it is MoE. Qwen3-30B-A3B reads 128 and 8, so 8 of the 128 experts in each layer run per token, while all 128 must be stored.- The model card’s own “N total / N activated” line if present, usually the fastest read. The Qwen3 cards state it; DeepSeek-V3.2’s card gives no parameter figures at all.
If fields 2 through 4 are missing on a card you want to run, treat it as unverified and either read config.json directly or pick a model whose card spells it out.
What this changes for the rest of the cluster
A few practical ripples for the other local-AI pages on this site. The VRAM arithmetic guide should be read with “total parameters = weight size” and “activated parameters = per-token compute” treated as separate columns; KV-cache size depends on layers, KV heads, head size and context length, not on the activated count. The inference-speed guide measures throughput in tokens per second, which for generation is driven by the bytes read per token (the activated weights) and memory bandwidth, not the total count. The family-picker hub is the right place to start when you have not decided whether MoE even applies to your workload.
Bottom line
MoE is a way to get a larger model’s total capacity at a smaller model’s per-token compute, paid for in memory: every expert must be stored in VRAM or system RAM. On a single desktop with one consumer GPU and enough system RAM, an MoE with a small activated count can run at speeds a dense model of the same total size cannot. A dense checkpoint is still simpler, is what the small tiers ship as, and gives the most model per byte when everything must fit in VRAM. Read the model’s config.json once before you commit a download.