Local AI by Editorial Team

Best Local LLM for Roleplay and Character Chat

Which open-weight models stay in character across long roleplays, what frontend to drive them with, and what the privacy picture looks like for intimate chats.

TL;DR

  • For most roleplay on a single consumer GPU, the practical picks are the Mistral-Small-3.x family (24B at 14-15 GB on disk, 32K or 128K context, Apache 2.0) plus the creative-writing fine-tunes built on it (Cydonia-24B-v4.3, Rocinante-X-12B-v1).
  • For very small hardware (8 GB or less), nemotron-mini at 4B / 2.7 GB is the only Ollama tag NVIDIA itself labels for “roleplay, RAG QA, and function calling”; the 4K context is the hard limit.
  • The frontend matters more than the model for long roleplays. SillyTavern, KoboldCpp, and text-generation-webui all read the same V2 character card spec and all run local. They differ in story mode, sampler controls, and how aggressive their world-info (lorebook) matching is.
  • Most “uncensored” local models are unfiltered because the system prompt is yours to write; the major safety framing lives in the prompt, not the weights. Treat the chat log like any other sensitive content on your disk.

Why “best for roleplay” is a separate question

A roleplay chat is not a short Q&A. The same model that answers a math problem in two paragraphs has to do four different jobs at once during a long scene:

  • Stay in character. Voice, vocabulary, mannerisms, and the rules of the world need to survive across thousands of turns.
  • Honor the lorebook. When the scene mentions a place or a magic system, the model has to surface the right lore entry without the writer pasting it in.
  • Honor the system prompt. The persona, the setting, the do-and-don’t list all sit in the system prompt. The model needs to treat it as binding without pushing back into a generic refusal.
  • Stay coherent across a long context. A 10K-token scene with five characters all talking differently is not the same workload as a 1K-token coding task.

A general chat model that is excellent at step-by-step math will still produce a flat, helpful-assistant character voice in a roleplay unless the system prompt does most of the work. The creative-writing fine-tunes and the “uncensored” instruction-tuned families are built to take a long character card and a tight system prompt and run with them.

This page picks one model per VRAM tier and pairs it with the right frontend. The privacy piece is its own section because roleplay chats often involve intimacy, identity work, or emotional content that does not belong on a vendor’s training cluster.

Smallest tier: 4B to 8B (8 GB or no GPU)

nemotron-mini (4B, 2.7 GB on disk, 4K context, commercial-friendly). The Ollama nemotron-mini page is explicit: this is the only small Ollama tag NVIDIA lists as optimized for “roleplay, RAG QA, and function calling.” It is 4B parameters, 2.7 GB on disk, with a 4,096-token context window. The trade-off is brutal: 4K context means you cannot run a long scene with a meaningful lorebook. It works for short chat-style roleplay and for evaluating whether the local stack is worth setting up at all.

Dolphin 3 (dolphin3:8b, 4.9 GB, 128K context). The Ollama dolphin3 page lists 8B parameters, 4.9 GB on disk, and a 128K context window. The README describes Dolphin as a series of instruct-tuned models “designed to be the ultimate general purpose local model, enabling coding, math, agentic, function calling, and general use cases” and explicitly frames the system owner as the one who controls “the system prompt,” “the alignment,” and “the guidelines.” This is the “uncensored local default” for many users: the model is steerable rather than pre-loaded with vendor alignment. For roleplay specifically, it is more obedient to a character card than a comparable general model, but it is not a creative-writing specialist.

Mistral-Small 3 (mistral-small:24b, 14 GB, 32K context, Apache 2.0). This is the older Mistral Small 3 tag and the workhorse tier; the smallest tier only goes up to 8 GB so most readers will land here next. Listed on the Ollama mistral-small page at 14 GB on disk with a 32K context window and Apache 2.0 licensing. The upstream Hugging Face model card for Mistral-Small-3.2-24B-Instruct-2506 lists 131,072 tokens of context (with the caveat that the Ollama tag uses 32K) and describes the model as a “minor update” of 3.1 with improved “instruction following, reduced repetition/infinite generations, and a more robust function calling template.” It is the most permissive licence for a frontier-feeling 24B local model, and it is the most common base for the roleplay fine-tunes below.

For users without a GPU at all, the CPU path is the same model with reduced context and slower tokens-per-second; the small tier (nemotron-mini, dolphin’s smaller sizes) is the only thing that responds in real time on CPU.

Mid tier: 12B to 24B (16 to 24 GB VRAM)

This is where most roleplay users actually land, because 24 GB is the cheapest VRAM that comfortably fits a creative-writing-tuned model with room left for context.

Mistral-Small 3.2 24B at 128K (mistral-small3.2:24b, 15 GB, 128K context). The Ollama mistral-small3.2 tag is the 128K-context variant of the Apache 2.0 family. This is the right base pick when you want the longest context for a deep lorebook without paying for a fine-tune. On a 24 GB card the 15 GB tag leaves ~9 GB for KV cache and OS overhead; on a 16 GB card the same tag pushes into disk offload.

Cydonia-24B-v4.3 (24B, Mistral-Small-3.2 base). The Hugging Face model card lists the lineage as Mistral-Small-3.1-24B-Base-2503 then Mistral-Small-3.2-24B-Instruct-2506 then Cydonia-24B-v4.3. Quantizations: GGUF, EXL3, iMatrix. The card’s evaluation vocabulary is “Creativity (writing, dynamism, imagination), Alignment (attitude, morality, formatting), Intelligence (adherence, knowledge, perception)” - the same roleplay-community criteria the creative-writing article on this site cites. The model card reads that “Foundational models today are optimized for non-creative uses, and I believe there is a place for AI in creativity and entertainment.” 2,438 downloads in the last month at the time of writing. The Hugging Face page does not restate the licence for the fine-tune; the Mistral-Small-3.x base is Apache 2.0, but downstream redistribution of generated prose under the fine-tune’s terms should be confirmed before commercial use.

Rocinante-X-12B-v1 (12B, BF16, Mistral-Nemo base). The Hugging Face model card identifies this as part of the author’s “4.0 gen” of roleplay-focused models, finetuned from Mistral-Nemo-Base-2407 through Mistral-Nemo-Instruct-2407. Recommended chat templates are Mistral v3 Tekken (with [SYSTEM_PROMPT] removed) or Metharme; the model supports <thinking></thinking> tags but also runs without them. The card describes focus areas as “creativity, dynamism, imagination, reduced (mis)alignment compared to corporate-tuned models, and adherence,” and acknowledges that “gpt-isms and recurring ‘it’s not just x, it’s y’ type formatting” still appear. 12B is small enough to run at usable speed on a 16 GB card with a long context.

Skyfall-31B-v4.2 (31B, Mistral-Small-3.x lineage via Magistral). Per the creative-writing article’s prior sourcing, the Hugging Face model tree for Skyfall-31B-v4.2 runs from Mistral-Small-3.1-24B-Base-2503 through Mistral-Small-3.2-24B-Instruct-2506 and Magistral-Small-2509. At 31B parameters it does not fit on a single 24 GB consumer card at full precision; it is a multi-GPU or heavily-quantized pick.

Larger tier: 30B to 70B (32 GB+ VRAM or workstation)

Two picks worth knowing at this tier, even if most roleplay users never reach them.

Mistral-Small 3.2 at full BF16. The Hugging Face card for Mistral-Small-3.2-24B-Instruct-2506 lists ~55 GB GPU RAM at BF16. This is the workstation pick; the 4-bit quant we use on a 24 GB card loses accuracy on subtle instruction-following, which matters when the roleplay needs the model to honor a tight “do not break character” rule.

Llama 3.3 70B (43 GB on disk, 128K context, Llama 3.3 Community License). The Ollama llama3.3 tag is 43 GB on disk with a 128K context window under the Llama 3.3 Community License, which the Ollama page notes “allows for commercial and research use in multiple languages.” It is the largest open-weight model that fits on a high-end Mac (64 GB unified memory) and on a 48 GB card without quantization tricks. For roleplay, the prose quality is the draw; the licence carries commercial-use thresholds that should be checked in the full licence text before any deployment above the personal-use scale.

The frontend layer

The frontend is where character cards, world info (lorebooks), prompt templates, sampler settings, and chat history all live. Three of the four local options are AGPL-3.0 and run entirely on the user’s machine.

SillyTavern (AGPL-3.0). Per the SillyTavern GitHub, it is “a locally installed user interface,” “free and open sourced,” with the explicit position “We do not provide any online or hosted services, nor programmatically track any user data.” 34.6k stars, 6.5k forks, 11,805 commits on the release branch, started as a fork of TavernAI 1.2.8 (MIT) in February 2023 with “over 300 contributors.” Backends include OpenAI, Claude, Mistral, OpenRouter, NovelAI, KoboldAI/CPP, Tabby, Horde, Ooba, and ComfyUI / Automatic1111 for image generation. Recommended local inference card: “a 3000-series NVIDIA graphics card with at least 6GB of VRAM.” SillyTavern is the standard V2 character-card frontend, with built-in WorldInfo (lorebook) matching, group chats, auto-summary of long chats, and the deepest lorebook tooling of the four.

KoboldCpp (AGPL-3.0). Per the KoboldCpp GitHub, it is “free and open-source software for running GGUF large language models (LLMs) on your own computer,” “One File. Zero Install,” and forked from ggml-org/llama.cpp. Bundled components retain their own licences (MIT for GGML, llama.cpp, stable-diffusion.cpp). Backends: CPU, NVIDIA CUDA, Vulkan (AMD/Nvidia/Intel), AMD ROCm, Apple Metal. Modes: chat, adventure, instruct, or story writing. The story-writing mode is the closest thing in the local-AI ecosystem to a purpose-built fiction interface; the single-file portability (no installer) makes it the easiest to set up on a writer’s existing laptop. KoboldCpp reads Tavern character cards from file or external sites and is the runtime inside most “KoboldAI-compatible” frontends.

Text Generation WebUI / TextGen (AGPL-3.0). Per the text-generation-webui GitHub, it is rebranded as “TextGen,” an open-source desktop app for running LLMs locally. Backends: llama.cpp, ik_llama.cpp, Transformers, ExLlamaV3, TensorRT-LLM. Modes: instruct, chat-instruct, and chat, with prompt templates in Jinja2. The “Notebook tab for free-form text generation outside of chat turns” is the key fiction feature. Exposes an OpenAI-compatible local API and an Anthropic-compatible one. 47.7k stars, 6.0k forks.

Open WebUI (OSS). The most general of the four. RAG over uploaded documents, multi-model chat, OpenAI-compatible, no character cards or world info. Right pick if the roleplay user also wants the same UI to drive a research chat and an everyday assistant.

All three of the roleplay-specific frontends read the Character Card V2 spec - spec: "chara_card_v2" with spec_version: "2.0", the required fields (name, description, personality, scenario, first_mes, mes_example, creator_notes, system_prompt, post_history_instructions, alternate_greetings, tags, creator, character_version, extensions), and the optional character_book lorebook with entries[] carrying keys, content, insertion_order, enabled, and optional secondary_keys, selective, constant, and position (before_char or after_char). A card saved in SillyTavern works in KoboldCpp and in text-generation-webui; that portability is the whole point of the spec.

What the lorebook does, and what it doesn’t

The lorebook (WorldInfo in SillyTavern, character_book in the V2 spec) is a list of trigger-key entries that get injected into the prompt when a relevant keyword appears in the conversation. Each entry has:

  • keys - the trigger words; if one appears in the recent context, the entry is eligible to inject.
  • content - the text to inject.
  • insertion_order - the priority when multiple entries match.
  • enabled - on/off toggle per entry.
  • secondary_keys - additional matchers (optional).
  • selective - if true, content is only injected when a secondary key also matches.
  • constant - if true, the entry is always injected regardless of keys.
  • position - before or after the character block.

This is the right primitive for worldbuilding. A scene set in a specific location pulls the relevant lorebook entries without the writer pasting them in. The hard limit is the front-end’s token_budget and scan_depth: a 200-entry world bible cannot all inject at once, and the matching algorithm is keyword-based, not semantic. Readers who want true RAG over a long lore document should pair a local model with Open WebUI’s RAG pipeline (semantic retrieval over uploaded files) rather than rely on lorebook keys.

The privacy piece, said out loud

Most roleplay chats include content a reader does not want stored on a vendor’s training cluster: intimate prompts, identity exploration, BDSM dynamics, suicidal ideation that gets worked through with a fictional character, kink negotiation, fictionalised confessions. The local stack is the only path that physically cannot leak those prompts, because the model runs on the user’s hardware and the chat log is on the user’s disk.

A few honest limits of that privacy:

  • The chat log is still a file. Back it up, encrypt it, or delete it like any sensitive document. A local model does not make the conversation ephemeral.
  • System prompts can leak. If the system prompt references real names, real locations, or real people, those strings get tokenized and sit in the KV cache. Treat the system prompt as sensitive too.
  • SillyTavern, KoboldCpp, and TextGen do not track. The SillyTavern GitHub explicitly says “We do not provide any online or hosted services, nor programmatically track any user data”; KoboldCpp’s README describes a single standalone executable with no required network calls. None of that removes the obligation to think about what you store on your disk.
  • A “local” frontend talking to a hosted endpoint is not local. SillyTavern can connect to OpenAI, Anthropic, NovelAI, and others. If the API endpoint is anything other than localhost or a local-network Ollama / llama.cpp server, every prompt leaves the building. This is the most common accidental leak in the local stack.

The choice of model affects privacy in one narrow way: the licence terms on generated prose matter if the conversation is published or used commercially. The base Mistral-Small-3.x family is Apache 2.0 per the Ollama mistral-small page; the fine-tunes inherit from a permissive base but the fine-tune author’s redistribution terms should be checked before commercial use.

What “uncensored” actually means here

“Local models are uncensored” is shorthand, not a property of the weights. The three things that determine what a local model will or will not refuse:

  1. The system prompt. This is the layer most roleplay users already control. A long character card with explicit do-and-don’t rules will push the model further than an empty system prompt.
  2. The instruction tune. Models trained with corporate safety framing (Llama, Claude, GPT) refuse more often. Models trained as instruct-tuned with steerable alignment (Dolphin, Mistral-Small, the roleplay fine-tunes) refuse less often. The Dolphin 3 page is the cleanest statement of this design.
  3. The sampler settings. Temperature, top-p, top-k, repetition penalty, and the presence/frequency penalty all change how willing the model is to keep going on a scene the alignment layer would rather not engage with. SillyTavern exposes all of these; text-generation-webui exposes them too.

There is no “local model that will do anything.” A 4B model with a tight refusal layer can refuse more than a 70B model with a permissive system prompt. The honest description is that local models are steerable in a way hosted endpoints are not.

Where local still loses

Three workloads still defeat consumer-card local models in late 2026:

  • Sub-4B models cannot sustain long character voice. A 4B model loses the character’s vocabulary by the second or third thousand tokens of a long scene. The first few hundred turns look fine; the rest turns into a flat helpful-assistant. Use nemotron-mini for short chat, not for novels-in-chats.
  • Lorebook scale is a hard limit. 200 lore entries with keyword matching can swallow context if the budget is generous, or fail to surface the right entry if the budget is tight. Long-form worldbuilding wants semantic RAG (Open WebUI) on top of the lorebook, not instead of it.
  • Frontier prose is still a hosted thing. A 24B local model writes competent prose; it does not write the paragraph-level work that a frontier hosted model does. For most roleplay that does not matter - the user wants a consistent character voice and a long lorebook, not a stylised paragraph. For users who want both, the local stack will not match the frontier hosted models on raw prose quality.

The bottom line

For a reader whose 24 GB consumer card is the runtime and whose hardware is not going to change:

  • The base model. Mistral-Small-3.2-24B at the Ollama mistral-small3.2 tag (15 GB, 128K context, Apache 2.0) for the long-context case, or Mistral-Small at the older Ollama mistral-small tag (14 GB, 32K context, Apache 2.0) for the lower-memory case.
  • The fine-tune if you want it. Cydonia-24B-v4.3 or Rocinante-X-12B-v1 from TheDrummer’s portfolio, both Mistral-based.
  • The frontend. SillyTavern for the deepest lorebook and group-chat features; KoboldCpp for a single-executable portable story mode; text-generation-webui for a notebook-tab free-form generator and OpenAI-compatible local API.
  • The privacy posture. Run the model on localhost, do not point the frontend at a hosted API, treat the chat log and the system prompt as sensitive files, and confirm the licence chain on the fine-tune if the prose is going commercial.

For 8 GB hardware, nemotron-mini is the only Ollama tag NVIDIA itself labels for roleplay; pair it with a tight lorebook and short scenes. For 16 GB, Rocinante-X-12B-v1 fits with room for context. For 32 GB and up, Mistral-Small-3.2 at BF16 is the cleanest baseline, with Llama 3.3 70B as the ceiling. The bytes stay on the machine, the system prompt is the safety layer, and the frontend is the one a roleplay user actually wants to run.