Local AI Out of Memory: How to Fix It (November 2026)
A local model that will not load is usually memory arithmetic. Decode the error strings, then cut context, KV cache, parallel requests and layers in order.
Tag
#llama-cpp
Local LLM Prompt Caching: Reuse Explained (October 2026)
How llama.cpp, vLLM and LM Studio decide whether to reuse your prompt's KV cache, what each reports about tokens saved, and where the reuse stops.
Why Your Local LLM Gives a Different Answer Each Time
Temperature, top-k, top-p and seed decide how random your local model is. What they do in Ollama, llama.cpp and vLLM, and how to get repeatable answers.
Local LLM Privacy: What Each Runtime Actually Sends Home
Side-by-side look at what Ollama, LM Studio, llama.cpp, vLLM, AnythingLLM, and Hugging Face TGI collect by default, and how to switch each one off.
Speculative Decoding: Drafts, EAGLE-3, n-gram (October 2026)
Speculative decoding strategies for local LLMs: how draft models, EAGLE-3, DFlash, and n-gram matching speed up llama.cpp and vLLM, and when each one helps.
Structured Output and Tool Use with Local LLMs (October 2026)
JSON mode, GBNF grammar-constrained decoding, and OpenAI-style tool calling across Ollama, llama-server, LM Studio, and vLLM.
Why AI Models Need a GPU: A Beginner Explainer (October 2026)
A plain-language walk through parallelism, matrix multiplication, VRAM, memory bandwidth, and why a CPU can run an LLM but rarely fast enough.
Surface Laptop Ultra and RTX Spark: Can It Run Local AI
Microsoft's first laptop on Nvidia's RTX Spark has a price and ship date. Here is what fits in 128GB of unified memory - and what it costs you.
Best Local AI for Creative Writing: Private Drafts
Run an LLM against your novel without sending an unpublished manuscript to a vendor. Models, front ends, and what the tradeoffs are.
What's New in Ollama 0.40, and Should You Upgrade?
v0.40.0-rc0 ships MLX-by-default on Apple Silicon, v0.35 adds decision models, transformers runs llama.cpp quants. What changed and how to upgrade.
MoE vs Dense Models for Local AI (September 2026)
What the 'A3B' and 'A22B' suffixes mean, when Mixture-of-Experts beats a dense model on your hardware, and how to read a model card to tell which is which.
Best Local Models by Context Window (October 2026)
Local LLMs ranked by maximum context length: 32K, 128K, 256K, and 1M+ tiers, with the model that fits each tier and the VRAM each one needs.
What Thinking Mode Actually Costs You (September 2026)
Reasoning tokens, latency, and KV cache cost of thinking mode in local LLMs. How to toggle it per runner and when it is the wrong choice.
Local LLM Backends Compared: Ollama, LM Studio, llama.cpp, vLLM
How Ollama, LM Studio, llama.cpp, and vLLM differ on model format, OpenAI-compatible API, hardware support, and which to pick for a home server.
Serve a Local Model to Your Team (September 2026)
Auth, reverse proxy, HTTPS, concurrency and audit: how a small team shares one local model without exposing it to the public internet.
Local AI When One GPU Isn't Enough (September 2026)
Multi-GPU options for self-hosted AI when one card runs out of room: how Ollama, llama.cpp and vLLM split models, and what the interconnects actually cost.
Run an LLM Locally on a Raspberry Pi (October 2026)
Raspberry Pi 4 and Pi 5 can run small open-weight models on CPU alone. What fits on 2 GB, 4 GB, 8 GB, or 16 GB of RAM, and what speed is realistic.
Best Used GPUs for Local AI in 2026 (September 2026)
The RTX 30 and 40 series cards that still clear current Ollama and llama.cpp floors, what VRAM tier each opens, and which variants are worth the used premium.
Serve a Local Model to Your Household (August 2026)
Practical patterns for letting two to six people in one home share one local model on one machine, with the right chat UI, bind address, and overlay network.
What Quantization Costs You in Quality (October 2026)
How much quality a smaller quant actually loses. Real MMLU and KL Divergence numbers across Q2_K through Q8_0, plus how to measure on your own workload.
Local AI on AMD and Intel GPUs (September 2026)
What works on AMD Radeon and Intel Arc hardware today, where the official ROCm and SYCL paths stop, and how Vulkan fills the gap.
Running Local LLMs on AMD and Intel GPUs in 2026
AMD's ROCm support splits by card and by OS, Intel archived both of its own LLM libraries this year, and llama.cpp is the path still standing.
What Quantization Actually Costs You in Quality
8-bit measures close to lossless. 4-bit ranges from a 0.6% gain to a 59% drop on one benchmark, depending on the model. The figures, fully attributed.
Local AI Without a GPU: What Actually Runs (October 2026)
Which open models run on CPU and RAM alone, how to size them, why mixture-of-experts helps, and what a machine with no discrete GPU cannot do.