What Thinking Mode Actually Costs You (September 2026)
Reasoning tokens, latency, and KV cache cost of thinking mode in local LLMs. How to toggle it per runner and when it is the wrong choice.
Tag
#local-ai
How much context can a local LLM actually hold? (September 2026)
The advertised context window is one number. What your GPU can actually serve is smaller. The KV cache math, RoPE scaling, and the practical ceiling.
How to Safely Download and Verify Open-Source LLMs
Sourced steps to safely download and verify an open-source LLM: safetensors, GPG-signed commits, pinned revisions, and HF / Ollama integrity checks.
Local LLM Backends Compared: Ollama, LM Studio, llama.cpp, vLLM
How Ollama, LM Studio, llama.cpp, and vLLM differ on model format, OpenAI-compatible API, hardware support, and which to pick for a home server.
How to Choose an Open-Weight Model Family (September 2026)
Qwen, Llama, Mistral, Gemma, DeepSeek, Phi - which family to commit to, what each is good at, and the licence traps that send you back to negotiate.
How to Evaluate Local LLMs on Your Own Workload (September 2026)
Why general benchmarks like MMLU and GPQA don't predict your results, the index-rebase trap, and a recipe for picking a local model from your own prompts.
Serve a Local Model to Your Team (September 2026)
Auth, reverse proxy, HTTPS, concurrency and audit: how a small team shares one local model without exposing it to the public internet.
Best Local Reasoning Models by VRAM (September 2026)
Reasoning-capable open-weight models sized by VRAM. DeepSeek-R1, Qwen3 with thinking, QwQ, Phi-4 reasoning, gpt-oss, and the licence traps.
Local AI When One GPU Isn't Enough (September 2026)
Multi-GPU options for self-hosted AI when one card runs out of room: how Ollama, llama.cpp and vLLM split models, and what the interconnects actually cost.
Fine-Tune a Local LLM with LoRA and QLoRA (September 2026)
How to adapt an open-weight model on your own hardware. What LoRA rank and alpha mean, when QLoRA's 4-bit NF4 is worth it, and how to measure it.
Run an LLM Locally on a Raspberry Pi (September 2026)
Raspberry Pi 4 and Pi 5 can run small open-weight models on CPU alone. What fits on 2 GB, 4 GB, 8 GB, or 16 GB of RAM, and what speed is realistic.
Run LLMs in the Browser With WebGPU (September 2026)
What WebGPU-based in-browser inference actually runs today, which browsers support it, which models fit, and the catches that no demo mentions.
Ollama's New Pricing: What the Credit-Pool Change Actually Means
Ollama swapped GPU-hour billing for per-token credits across Pro, Max and Team plans. What the tiers cost, what's free, and how the no-logging promise holds up.
Best Used GPUs for Local AI in 2026 (September 2026)
The RTX 30 and 40 series cards that still clear current Ollama and llama.cpp floors, what VRAM tier each opens, and which variants are worth the used premium.
Serve a Local Model to Your Household (August 2026)
Practical patterns for letting two to six people in one home share one local model on one machine, with the right chat UI, bind address, and overlay network.
Open Model Licences for Commercial Use (August 2026)
Apache-2.0, MIT, Qwen variants, Llama community terms: what the Hugging Face API actually returns and the per-repo traps that send you back to negotiate.
What Quantization Costs You in Quality (August 2026)
How much quality a smaller quant actually loses. Real MMLU and KL Divergence numbers across Q2_K through Q8_0, plus how to measure on your own workload.
Local AI on AMD and Intel GPUs (August 2026)
What works on AMD Radeon and Intel Arc hardware today, where the official ROCm and SYCL paths stop, and how Vulkan fills the gap.
How much VRAM does a local LLM actually need? (August 2026)
The published GGUF file size is the weights. VRAM use also includes the KV cache, framework overhead, and your context length. The math, walked through.
Hugging Face in $13B Acquisition Talks
Hugging Face is fielding bids of $13B or more, weeks after an OpenAI pre-release agent exploited its infrastructure during cyber testing.
When Local AI Beats the API: The Cost Crossover (August 2026)
The monthly token volume at which a self-hosted model becomes cheaper than paying per token to GPT-5, Claude, or Gemini, and the workloads that never get there.
What It Takes to Run Qwen3.8-27B on Your Own Hardware
Five stock GGUF quants for Qwen3.8-27B span 9.01 GB to 29.05 GB. A file size is not a VRAM requirement, and a 262,144-token ceiling is not free to fill.
What to Check Before Buying a Used GPU for Local AI
The compute capability, driver and ROCm facts to verify on a second-hand card before it has to run Ollama, llama.cpp or a current CUDA toolkit.
Local vs API Cost: Work Out Your Own Crossover Point
There is no single crossover figure. The cost components on each side, the arithmetic shape, and why every price in it carries a date.