Speakers of languages with weak model support often get worse answers from AI models than English speakers do. Local inference is one way to work around that, but only if the model you pick actually handles your language well - and the headline language counts on Hugging Face cards are not a quality guarantee.
The practical choice is between four open-weight families with published model cards. They differ on licence, gating, the size they ship in, and the quality they deliver outside English. The table below is built from what each model’s card and the Hugging Face API actually say, not from the language count claim alone.
What the language count on a card actually means
A model card often opens with a large number (“201 languages”, “140+ languages”, “24 languages”). That number tells you how many languages the training set at least touched, not how well the model handles any one of them. A model that lists 201 languages will still type-mix or romanise a low-resource language it rarely saw during pretraining.
Two practical signals to check before trusting a number:
- Tokenisation fragmentation. SentencePiece or BPE vocabularies built primarily on English split a Hindi word into four sub-tokens where a model trained on Hindi might split it into one. The more sub-tokens per word, the more context a non-English message eats per real word, and the harder the model finds the next-token prediction. Phi-4-mini’s card lists 23 named supported languages; Qwen3.5’s card claims support for “201 languages and dialects”.
- Benchmark depth. Qwen reports multilingual-specific numbers for Qwen3.5-9B: MMMLU 81.2 across many languages, WMT24++ 72.6 over 55 languages via XCOMET-XXL, PolyMATH 57.3. Microsoft reports Multilingual MMLU 49.3 and MGSM 63.9 for Phi-4-mini. Gemma 3’s card publishes a multilingual table for its pre-trained (not instruction-tuned) 4B model: MGSM 34.7, Global-MMLU-Lite 57.0, WMT24++ (ChrF) 48.4. All of these are vendor-reported figures, not independent tests.
The four models worth comparing
Three of the four have an official Ollama library tag. Mistral Small 4 does not: as of 2026-10-11 the Ollama mistral-small library tags point to the older 22B and 24B Mistral Small releases, and there is no mistral-small4 library page. The licence and gating flags below come from the Hugging Face API (cardData.license, gated); Ollama sizes are as listed on the Ollama tags pages on 2026-10-11.
| Model | Params | Licence | Gated? | Ollama tag (default q4_K_M) | Native context | Languages claimed | Multilingual benchmark |
|---|---|---|---|---|---|---|---|
| Qwen3.5-9B | 9B | Apache-2.0 | No | qwen3.5:9b-q4_K_M (6.6 GB; the default qwen3.5:9b tag is listed at 6.6-7.6 GB) | 262K (extensible to 1,010K via YaRN) | “201 languages and dialects” | MMMLU 81.2, WMT24++ 72.6 (55 langs, XCOMET-XXL), PolyMATH 57.3 |
| Mistral Small 4 119B A6B | 119B total / 6.5B activated | Apache-2.0 | No | None in the Ollama library (mistral-small:latest is the older 24B model, 14 GB, 32K context) | 256K (--max-model-len 262144) | 24 language codes in the card metadata; card text says “dozens of languages” | None published; the card’s AA LCR comparison (0.72 with 1.6K characters of output vs 5.8-6.1K for Qwen models) is a long-context reasoning and output-length result, not a multilingual one |
| Gemma 3 4B IT | 4B | gemma (custom) | Manual - must accept Google’s terms at ai.google.dev/gemma/terms | gemma3:4b (3.4 GB) | 128K in / 8K out | ”over 140 languages” | Pre-trained 4B: MGSM 34.7, Global-MMLU-Lite 57.0, WMT24++ (ChrF) 48.4 |
| Phi-4-mini-instruct | 3.8B | MIT | No | phi4-mini:3.8b (2.5 GB) | 128K | 23 named languages (Arabic, Chinese, Czech, Danish, Dutch, English, Finnish, French, German, Hebrew, Hungarian, Italian, Japanese, Korean, Norwegian, Polish, Portuguese, Russian, Spanish, Swedish, Thai, Turkish, Ukrainian) | Multilingual MMLU 49.3, MGSM 63.9 |
The licence matters here. Apache-2.0 and MIT both allow commercial use with attribution; Apache-2.0 also carries an explicit patent grant, MIT does not. Gemma’s custom licence must be accepted on Hugging Face before gated access unlocks (gated: "manual" in the API), and Google’s terms add a list of prohibited uses that the Apache-2.0 and MIT licences do not have. MIT is the shortest of the four licences.
Pick by task
Cross-language chat and translation, with one machine. Qwen3.5-9B on a 12 GB GPU is the broadest fit. Its 9B parameters are small enough to run at q4_K_M (6.6 GB) on consumer hardware, and its card publishes a vendor-reported WMT24++ score of 72.6 averaged over 55 languages (XCOMET-XXL). The 262K native context can hold a long bilingual transcript in a single prompt. Internal link: see best-local-llm-runners for the Ollama vs llama.cpp vs LM Studio question, and local-ai-power-and-thermals for what a 9B model at q4_K_M draws from the wall.
Production summarisation where output length is metered. Mistral Small 4 119B A6B is a mixture-of-experts model - 119B total, 6.5B active per token. Its card reports an AA LCR (Artificial Analysis long-context reasoning) score of 0.72 with 1.6K characters of output, against 5.8-6.1K characters for Qwen models at comparable scores. That is a short-output claim, not a multilingual benchmark. It is also not a laptop model: the Ollama mistral-small:latest tag (14 GB) is the older 24B Mistral Small, not this one, and even at 4 bits per weight Small 4’s 119.4 billion parameters come to about 60 GB (56 GiB) of weights before any KV cache. Internal link: pair it with moe-vs-dense-local-ai for the parameter math behind “active per token.”
Low-resource languages with a 4-8 GB GPU. Gemma 3 4B IT. The card claims “multilingual support in over 140 languages” and the model takes text and image input. The catch is the licence: gated: "manual" on Hugging Face means a one-time click-through acceptance of Google’s Gemma terms, and the card’s multilingual scores are for the pre-trained 4B model rather than the instruction-tuned one. For image-heavy work (signs, scanned forms, multilingual captions) it is the smallest image-capable option of the four at 3.4 GB (Phi-4-mini is smaller but text-only). If you would rather skip the gating step and have the memory, Qwen3.5-9B is the larger alternative.
Edge hardware and MIT-licensed deployment. Phi-4-mini-instruct. The 23 named languages are mostly European plus Arabic, Chinese, Hebrew, Japanese, Korean and Thai - not as broad as Qwen3.5 or Gemma. The card also warns that the model is “trained primarily on English text” and that other languages “will experience worse performance”. At 2.5 GB the q4_K_M build fits a Raspberry Pi 5 with 8 GB of RAM (see run-llm-locally-on-raspberry-pi) and the MIT licence means there is no gating, no separate terms page, and no use-case prohibition list to read. Its vendor-reported MGSM 63.9 (multilingual grade-school math) is the figure to look at if your workload leans on reasoning in non-English prompts.
Practical setup and the catches
There are three recurring pitfalls when running these models for non-English work.
- System prompt language sets the response language. A model that handles both English and Japanese will still drift to whichever language the system prompt or the first user turn uses. For bilingual deployments, set the system prompt in the target language and front-load an example turn in the same language; do not rely on a single mid-prompt switch.
- Quantisation quality can vary by language. A Q4_K_M build loses some quality against BF16, and none of the four cards publishes per-language quantisation results, so the loss in less-trained languages is unmeasured here. If your workload is a single European language plus English, Q4_K_M is fine. If it includes low-resource languages with light training data, drop to a higher quant (Q5_K_M, Q8_0) or run BF16 if you have the VRAM.
- Tokeniser cost adds up at long context. A sentence that a tokenizer splits into many more sub-tokens than its English equivalent eats proportionally more KV cache. For very long-document Q&A over non-English material, best-local-models-by-context-window explains what each family’s KV-cache math actually looks like.
Bottom line
- Need one model that handles the most languages on consumer hardware: Qwen3.5-9B at
qwen3.5:9b-q4_K_M(6.6 GB, Apache-2.0, no gating, 262K context, multilingual benchmarks over 55 languages). - Need short outputs on long inputs and have server-class memory: Mistral Small 4 119B A6B (no official Ollama tag; about 60 GB of weights at 4-bit, Apache-2.0, 119B/6.5B MoE, 256K context; its card publishes no multilingual scores).
- Image-heavy multilingual workloads on small hardware, and you can accept the gating: Gemma 3 4B IT at
gemma3:4b(3.4 GB, custom licence, 140+ languages, vision-language). - Edge deployment in 23 named languages, MIT licence: Phi-4-mini-instruct at
phi4-mini:3.8b(2.5 GB, MIT, no gating, 128K context). - Avoid choosing on the headline language count alone. The published multilingual benchmarks (MMMLU, WMT24++, MGSM, PolyMATH, Global-MMLU-Lite) and the specific named-language list tell you more about whether your real workload will work.