Local AI Out of Memory: How to Fix It (November 2026)
A local model that will not load is usually memory arithmetic. Decode the error strings, then cut context, KV cache, parallel requests and layers in order.
Category
Local AI
How much disk space do local AI models need? (November 2026)
Where local AI models live on disk, what each model costs at every quantization, why the same weights get stored three times over, and how to reclaim the space.
Mistral Le Chonk: 1 Trillion Parameters, 8-GPU Recipe
Mistral released the 1T open-weight Le Chonk. The official serving recipe assumes 8 GPUs and the smallest GGUF is 233 GB, so the API is the realistic route.
Run LLMs Locally on Linux and Windows (October 2026)
How to install Ollama on Linux and Windows, run a first model, confirm the GPU is being used, and fix the setup errors that stop a first run.
Local LLM Prompt Caching: Reuse Explained (October 2026)
How llama.cpp, vLLM and LM Studio decide whether to reuse your prompt's KV cache, what each reports about tokens saved, and where the reuse stops.
Driving Claude Code With Local Ollama Models (October 2026)
Which open-weight models hold up behind Claude Code, how Ollama's Anthropic-compatible endpoint connects them, and what each Ollama tag costs to run.
Why Your Local LLM Gives a Different Answer Each Time
Temperature, top-k, top-p and seed decide how random your local model is. What they do in Ollama, llama.cpp and vLLM, and how to get repeatable answers.
Three Open-Weight Drops in a Week: What Each One Runs On
Muse Spark Open, Reflection Beam, and Granite Switch 4.1 all landed on Hugging Face within two days. What fits on one GPU, a Mac, or a cluster.
Best Local LLM for Roleplay and Character Chat
Which open-weight models stay in character across long roleplays, what frontend to drive them with, and what the privacy picture looks like for intimate chats.
Best vector databases for self-hosted RAG (October 2026)
Six self-hosted vector databases for RAG compared: Qdrant, Milvus, Chroma, Weaviate, pgvector, LanceDB. Licence, indexing, the catch.
Best Local LLM for Math and Quantitative Reasoning
Which open-weight models actually solve math on your hardware, from quick arithmetic to AIME-level problems, ranked by VRAM tier and use case.
Best Local Chat Models by VRAM Tier (October 2026)
Open-weight chat LLMs by VRAM tier: Qwen3.5, Qwen3.6, Gemma 4 and gpt-oss picks, with Ollama tag sizes and licences.
Local LLM guardrails compared (October 2026)
Llama Guard, Prompt Guard, NeMo Guardrails, guardrails-ai, and llm-guard: a feature-by-feature comparison of options for adding guardrails to a self-hosted LLM.
Anthropic: Small AI Models Could Self-Replicate on Your IoT
Anthropic's red team says small open models may be months from acting as self-replicating worms on IoT such as $55 TV boxes. What self-hosters need to know.
Best Local LLM for SQL Query Generation
Self-hosted models that turn plain-English questions into SQL for PostgreSQL, MySQL, SQLite, and DuckDB, with BIRD-SQL results, licences, and RAM needs.
Speculative Decoding: Drafts, EAGLE-3, n-gram (October 2026)
Speculative decoding strategies for local LLMs: how draft models, EAGLE-3, DFlash, and n-gram matching speed up llama.cpp and vLLM, and when each one helps.
Structured Output and Tool Use with Local LLMs (October 2026)
JSON mode, GBNF grammar-constrained decoding, and OpenAI-style tool calling across Ollama, llama-server, LM Studio, and vLLM.
Best Ollama Models to Drive Claude Code Without Anthropic's API
Which Ollama models work as a local backend for Claude Code's CLI, what features degrade without Anthropic's API, and how to wire it up.
Why AI Models Need a GPU: A Beginner Explainer (October 2026)
A plain-language walk through parallelism, matrix multiplication, VRAM, memory bandwidth, and why a CPU can run an LLM but rarely fast enough.
Surface Laptop Ultra and RTX Spark: Can It Run Local AI
Microsoft's first laptop on Nvidia's RTX Spark has a price and ship date. Here is what fits in 128GB of unified memory - and what it costs you.
Best Local LLM for Multilingual Use (October 2026)
Which open-weight model actually handles non-English work, what each provider claims about language coverage, and the tradeoffs between breadth and depth.
Best Local Vision Models by VRAM (October 2026)
Open-weight vision-language models sized by VRAM: Qwen3-VL 2B to 235B-A22B, Mistral Small 3.2, Gemma 3, Pixtral, and where each tier breaks.
Best Local LLM for Spreadsheet Analysis on a Laptop
Practical tier guide to running a local LLM on Excel, CSV, and tabular data. Picks by RAM, JSON output for clean rows, and what to skip.
Self-Host a Private AI Home Server: A Primer (October 2026)
How to pick the hardware, OS, and runtime for a 24/7 local AI server, from a low-power mini PC to a used 3090 workstation, and what each costs to run.