Local AI Out of Memory: How to Fix It (November 2026)
A local model that will not load is usually memory arithmetic. Decode the error strings, then cut context, KV cache, parallel requests and layers in order.
Tag
#local-ai
How much disk space do local AI models need? (November 2026)
Where local AI models live on disk, what each model costs at every quantization, why the same weights get stored three times over, and how to reclaim the space.
Run a Local LLM on a NAS: Synology and QNAP Setup
Run Ollama in a container on a Synology or QNAP NAS: model storage, memory limits, context sizing, and reaching it from your laptop and phone.
Mistral Le Chonk: 1 Trillion Parameters, 8-GPU Recipe
Mistral released the 1T open-weight Le Chonk. The official serving recipe assumes 8 GPUs and the smallest GGUF is 233 GB, so the API is the realistic route.
Run LLMs Locally on Linux and Windows (October 2026)
How to install Ollama on Linux and Windows, run a first model, confirm the GPU is being used, and fix the setup errors that stop a first run.
How to Set Up a Local Voice Assistant With Home Assistant
Home Assistant can run a voice assistant with nothing leaving the house: Whisper or Speech-to-Phrase for speech, Piper for replies, a local LLM for the brain.
Host an AI Discord or Telegram Bot on Your Own Server
Run a private Discord or Telegram chatbot on a local model: wiring, token handling, the intents it needs, rate limits, and a systemd unit.
Local LLM Prompt Caching: Reuse Explained (October 2026)
How llama.cpp, vLLM and LM Studio decide whether to reuse your prompt's KV cache, what each reports about tokens saved, and where the reuse stops.
How to Run Ollama in Docker So the GPU Still Works
Containerize Ollama without losing GPU acceleration: the NVIDIA toolkit setup, the AMD device flags, the volume that keeps your models, and how to check.
Driving Claude Code With Local Ollama Models (October 2026)
Which open-weight models hold up behind Claude Code, how Ollama's Anthropic-compatible endpoint connects them, and what each Ollama tag costs to run.
Why Your Local LLM Gives a Different Answer Each Time
Temperature, top-k, top-p and seed decide how random your local model is. What they do in Ollama, llama.cpp and vLLM, and how to get repeatable answers.
Three Open-Weight Drops in a Week: What Each One Runs On
Muse Spark Open, Reflection Beam, and Granite Switch 4.1 all landed on Hugging Face within two days. What fits on one GPU, a Mac, or a cluster.
Best Local LLM for Roleplay and Character Chat
Which open-weight models stay in character across long roleplays, what frontend to drive them with, and what the privacy picture looks like for intimate chats.
Best vector databases for self-hosted RAG (October 2026)
Six self-hosted vector databases for RAG compared: Qdrant, Milvus, Chroma, Weaviate, pgvector, LanceDB. Licence, indexing, the catch.
Best Local LLM for Math and Quantitative Reasoning
Which open-weight models actually solve math on your hardware, from quick arithmetic to AIME-level problems, ranked by VRAM tier and use case.
Best Local Chat Models by VRAM Tier (October 2026)
Open-weight chat LLMs by VRAM tier: Qwen3.5, Qwen3.6, Gemma 4 and gpt-oss picks, with Ollama tag sizes and licences.
Local LLM guardrails compared (October 2026)
Llama Guard, Prompt Guard, NeMo Guardrails, guardrails-ai, and llm-guard: a feature-by-feature comparison of options for adding guardrails to a self-hosted LLM.
Local LLM Privacy: What Each Runtime Actually Sends Home
Side-by-side look at what Ollama, LM Studio, llama.cpp, vLLM, AnythingLLM, and Hugging Face TGI collect by default, and how to switch each one off.
Anthropic: Small AI Models Could Self-Replicate on Your IoT
Anthropic's red team says small open models may be months from acting as self-replicating worms on IoT such as $55 TV boxes. What self-hosters need to know.
Best Local LLM for SQL Query Generation
Self-hosted models that turn plain-English questions into SQL for PostgreSQL, MySQL, SQLite, and DuckDB, with BIRD-SQL results, licences, and RAM needs.
Speculative Decoding: Drafts, EAGLE-3, n-gram (October 2026)
Speculative decoding strategies for local LLMs: how draft models, EAGLE-3, DFlash, and n-gram matching speed up llama.cpp and vLLM, and when each one helps.
How to Use a Local LLM With Obsidian, Joplin, Logseq, and Notion
Plug a local Ollama model into Obsidian, Joplin, Logseq, or Notion so notes never leave your machine. Plugins that work, ones that do not, and what each costs.
Structured Output and Tool Use with Local LLMs (October 2026)
JSON mode, GBNF grammar-constrained decoding, and OpenAI-style tool calling across Ollama, llama-server, LM Studio, and vLLM.
Best Local LLM for Proofreading and Editing Your Writing
Run a proofreading LLM on your laptop so drafts, manuscripts, and sensitive documents never leave your machine. Models, front ends, and what each tier gets you.