Best Local LLM for Tabletop Game Masters

Which local model to run for NPC dialogue, session prep and rules lookup, with per-VRAM picks, exact file sizes and the licence caveats for published notes.

A game master runs four separate jobs on one machine, and they fail in different ways: an NPC whose voice drifts once the scene runs long, a recap that drops the one clue that mattered, a rules answer the model half-remembers, and a plot hook that reads like last month’s hook. The model that is good at one of those is often wrong for the other three.

Every number below is a byte count from a Hugging Face file tree, a licence field or a config value from repository metadata, or a default quoted from that project’s documentation. Nothing here comes from a benchmark run.

The four jobs, and what each one needs

JobWhat it needsHow it goes wrong
NPC dialogue at the tableA chat frontend, a card per NPC, a voice that holds over a long contextIt narrates for you, forgets who is speaking, or answers as an assistant
Session prep and recapsInstruction following and summarising over notes you paste inInvented lore, or a recap that omits the thread you needed
Rules lookup over the group’s PDFsRetrieval over your own documents, not model memoryA confident answer to a rule that is not in the book
Plot hooks and handoutsCreative writing with format controlEvery hook comes out the same shape

The picks, by memory tier

These are creative and roleplay fine-tunes, because a general assistant model asked to roleplay produces a helpful-assistant voice dressed up as a dragon. All five come from one fine-tuning author whose model cards name a base model for each, and the sizes are the Q4_K_M and Q8_0 builds from the GGUF repositories those cards link to.

VRAMPickQ4_K_M bytesQ8_0 bytesBase modelBase licence
8 GBAnubis-Mini-8B-v15,073,837,312 (4.73 GiB)8,540,784,864 (7.95 GiB)Llama 3.3 8Bllama3.3
12 GBRocinante-XL-16B-v110,063,921,216 (9.37 GiB)17,078,321,216 (15.91 GiB)Mistral Nemo 12BApache 2.0
16 GBCydonia-24B-v4.314,333,910,432 (13.35 GiB)25,054,780,832 (23.33 GiB)Mistral Small 3.2 24BApache 2.0
24 GBOrion-26B-A4B-v117,035,039,776 (15.87 GiB)26,859,861,024 (25.02 GiB)Gemma 4 26B-A4BApache 2.0
32 GBArtemis-31B-v1.219,517,721,024 (18.18 GiB)32,635,677,120 (30.39 GiB)Gemma 4 31BApache 2.0

To run a general model instead, the cluster’s usual picks apply: qwen3.5:9b-q4_K_M and gemma4:e4b-it-q4_K_M at 6.6 GB on the small end, gpt-oss:20b at 14 GB, and qwen3.8:27b-q4_K_M at 18 GB or qwen3.8:27b-q8_0 at 30 GB at the top, all on the Ollama library tag pages. Safer for rules reasoning, weaker for voice. The VRAM tier comparison has the free-memory arithmetic for each of those tags.

The catch on each pick

Anubis-Mini-8B-v1 is the only one that fits a laptop with no discrete GPU to spare, and its card states the trade-off: “While intelligence, correctness, and problem solving are not my priority, they are still one of many qualities I want in my models.” Its base is a Llama 3.3 8B fine-tune under the llama3.3 community licence, not Apache 2.0.

Rocinante-XL-16B-v1 is a Mistral Nemo 12B base, which the upstream config.json puts at 131,072 max positions. Its Q4_K_M build leaves about 2.63 GiB on a 12 GiB card, too little for a long scene.

Cydonia-24B-v4.3 is the one most groups should look at first. Its card’s “What’s New?” list reads “Much better roleplay”, “Might refuse/be more positive”, “But much better roleplay”, which is the trade in one line: it refuses more than the older tunes.

Orion-26B-A4B-v1 is a mixture-of-experts tune of Gemma 4 26B-A4B, which the base card puts at 25.2B total and 3.8B active parameters, explaining that ‘The “A” in 26B A4B stands for “active parameters” in contrast to the total number of parameters the model contains.’ It behaves like a small model for speed while carrying a large model’s weight, which makes it the comfortable 24 GB pick.

Artemis-31B-v1.2 is the quality ceiling on this list and the fussiest. Its card warns “Sampler wrangling may be required (or be conservative)”. At Q8_0 the weights alone take 30.39 GiB, leaving nothing on a 32 GB card for a long context.

The context you actually get

The KV cache decides whether a scene survives. Computed from each base model’s own config.json at a 16-bit cache. Where a model mixes sliding-window and full attention, only the full-attention layers grow with the scene, so those rows count the full-attention layers alone:

PickKV per tokenCost of a 32K-token scene
Anubis-Mini-8B-v1128 KiB (131,072 bytes)4 GiB
Rocinante-XL-16B-v1160 KiB (163,840 bytes)5 GiB
Cydonia-24B-v4.3160 KiB (163,840 bytes)5 GiB
Orion-26B-A4B-v120 KiB (20,480 bytes)0.625 GiB
Artemis-31B-v1.280 KiB (81,920 bytes)2.5 GiB

Orion’s advantage is structural: Gemma 4 26B-A4B interleaves sliding-window attention with full attention on 5 of its 30 layers, so only those five carry a cache that grows with the scene, and each of them holds 2 key-value heads of 512 dimensions. Gemma 4 31B, the base under Artemis, does the same across 10 full-attention layers of 60. On a 24 GB card Cydonia leaves about 10.65 GiB after the weights and Orion about 8.13 GiB, but Orion spends 0.625 GiB of that on a 32K scene where Cydonia spends 5 GiB.

Loading a GGUF, and setting the context length

None of these five is in the Ollama library: the model index lists no cydonia, anubis, artemis, orion or roc entry. Load the GGUF directly instead. Ollama’s import documentation states that “Ollama does not quantize GGUF models during import”, and a single-file GGUF needs a Modelfile with FROM /path/to/file.gguf followed by ollama create my-model. llama.cpp’s server takes -c, --ctx-size N and --port (default 9931).

The window matters as much as the weights. Ollama picks the context from your VRAM, and its documentation states the defaults as ”< 24 GiB VRAM: 4k context”, “24-48 GiB VRAM: 32k context” and ”>= 48 GiB VRAM: 256k context”. Four thousand tokens is the default on 8, 12 and 16 GB cards, short enough that a scene runs out mid-conversation. The same page notes that “Tasks which require large context like web search, agents, and coding tools should be set to at least 64000 tokens”, and shows OLLAMA_CONTEXT_LENGTH=64000 ollama serve. Check what you got with ollama ps.

The frontend: cards, lorebooks and recaps

SillyTavern is the piece that turns a model into a table. The current release is 1.19.1, published on 2026-11-19, and the roleplay and character chat guide covers the card format in more depth.

Each NPC is a character card. The group chat feature keeps one card per member and adds “Member Management”, where characters are added, removed, muted or reordered. The “Reply order strategies” setting decides who speaks: “Manual” lets you pick, “Natural Order” detects names in the latest message, “Pooled Order” picks someone who has not spoken since your last message. By default only the active speaker’s card is in context under “Swap character cards”; “Join character cards” merges every member’s description into one prompt, which risks blending the characters together.

World Info is the lorebook. Each entry has a “Key” that triggers it and an “Entry Content” that the documentation defines as “The text that is inserted into the prompt upon entry activation.” “Insertion Position” decides where the text lands, with options including “Before Char Defs” and “After Char Defs”. “Scan Depth” controls how many recent messages are examined, and “Context % / Budget” caps the tokens World Info may consume, with “Alert on overflow” to warn you when it goes over. Entries carry “Character Filter” to restrict activation to a named NPC, and timed effects (“Sticky”, “Cooldown”, “Delay”) measured in messages.

Recaps are handled by the Summarize extension, which writes a running summary into the chat metadata. The docs are blunt about the quality: summaries “may lose some important details or contain hallucinations”, so the state needs checking by hand. Of the three build modes, “Classic, blocking” is the one recommended for llama.cpp and similar backends. The Extras API path defaults to Qiliang/bart-large-cnn-samsum-ChatGPT_v3, which has “a very small context size (~1024 tokens)”.

Rules lookup over the group’s own PDFs

A lorebook keyed on names and places is not retrieval over a 300-page rulebook. For that, pair a local model with a stack that embeds your PDFs and reranks the chunks. The comparison of six self-hosted RAG stacks covers the licensing and reranking differences, and the embedder to point them at is Qwen3-Embedding-0.6B, which the Hugging Face API lists as Apache 2.0 and ungated and which the embedding and reranker VRAM guide picks as the default. Keep the two apart: World Info for what the fiction knows, retrieval for the rulebook.

What you can publish, and what the licences say

Two questions sit behind publishing session notes or a paid adventure.

The first is the model licence. Three of the five base models here are Apache 2.0 (Mistral Nemo, Mistral Small 3.2, Gemma 4), which impose no revenue or user-count conditions. The Llama 3.3 base behind Anubis-Mini is under the llama3.3 community licence, which carries its own conditions.

The second is the fine-tune. The Hugging Face API returns no licence field for any of the five fine-tune repositories, and their model card frontmatter lists only base_model, so the base licence is verifiable while the fine-tune’s own terms are not stated. Read those terms before publishing. What the game publisher permits in your notes is not covered by any model licence.

Bottom line

  • 8 GB: Anubis-Mini-8B-v1 at Q4_K_M, with a 16K context. Voice is the weakest part, and the base licence is Llama 3.3 community, not Apache 2.0.
  • 12 GB: Rocinante-XL-16B-v1 at Q4_K_M, leaving room for about 16K tokens of scene.
  • 16 GB: Cydonia-24B-v4.3 at Q4_K_M. The best-balanced voice here, though it refuses more than the older tunes. A 24 GB card is needed for a 32K scene.
  • 24 GB: Orion-26B-A4B-v1 at Q4_K_M. The mixture-of-experts base makes it the fastest of the five and the cheapest in context.
  • 32 GB: Artemis-31B-v1.2 at Q4_K_M for a long scene, or at Q8_0 for quality on short scenes only.
  • The setup: one character card per NPC in SillyTavern, World Info entries for locations and factions, a retrieval stack over the rulebook PDFs, and OLLAMA_CONTEXT_LENGTH set explicitly, because the default under 24 GiB is 4k tokens.

The bytes stay on your machine, the character card is what makes a model sound like a person, and the retrieval stack is what stops it inventing your rules. More on the hardware side, from picking a model to the context it can hold, is collected under Local AI.

Other GPUs and tasks: Local AI by VRAM · RSS