Why AI Models Need a GPU: A Beginner Explainer (October 2026)
A plain-language walk through parallelism, matrix multiplication, VRAM, memory bandwidth, and why a CPU can run an LLM but rarely fast enough.
Tag
#llama-cpp
Surface Laptop Ultra and RTX Spark: Can It Run Local AI
Microsoft's first laptop on Nvidia's RTX Spark has a price and ship date. Here is what fits in 128GB of unified memory - and what it costs you.
Best Local AI for Creative Writing: Private Drafts
Run an LLM against your novel without sending an unpublished manuscript to a vendor. Models, front ends, and what the tradeoffs are.
What's New in Ollama 0.40, and Should You Upgrade?
v0.40.0-rc0 ships MLX-by-default on Apple Silicon, v0.35 adds decision models, transformers runs llama.cpp quants. What changed and how to upgrade.
MoE vs Dense Models for Local AI (September 2026)
What the 'A3B' and 'A22B' suffixes mean, when Mixture-of-Experts beats a dense model on your hardware, and how to read a model card to tell which is which.
Best Local Models by Context Window (September 2026)
Local LLMs ranked by maximum context length: 32K, 128K, 256K, and 1M+ tiers, with the model that fits each tier and the VRAM each one needs.
What Thinking Mode Actually Costs You (September 2026)
Reasoning tokens, latency, and KV cache cost of thinking mode in local LLMs. How to toggle it per runner and when it is the wrong choice.
Local LLM Backends Compared: Ollama, LM Studio, llama.cpp, vLLM
How Ollama, LM Studio, llama.cpp, and vLLM differ on model format, OpenAI-compatible API, hardware support, and which to pick for a home server.
Serve a Local Model to Your Team (September 2026)
Auth, reverse proxy, HTTPS, concurrency and audit: how a small team shares one local model without exposing it to the public internet.
Local AI When One GPU Isn't Enough (September 2026)
Multi-GPU options for self-hosted AI when one card runs out of room: how Ollama, llama.cpp and vLLM split models, and what the interconnects actually cost.
Run an LLM Locally on a Raspberry Pi (September 2026)
Raspberry Pi 4 and Pi 5 can run small open-weight models on CPU alone. What fits on 2 GB, 4 GB, 8 GB, or 16 GB of RAM, and what speed is realistic.
Best Used GPUs for Local AI in 2026 (September 2026)
The RTX 30 and 40 series cards that still clear current Ollama and llama.cpp floors, what VRAM tier each opens, and which variants are worth the used premium.
Serve a Local Model to Your Household (August 2026)
Practical patterns for letting two to six people in one home share one local model on one machine, with the right chat UI, bind address, and overlay network.
What Quantization Costs You in Quality (September 2026)
How much quality a smaller quant actually loses. Real MMLU and KL Divergence numbers across Q2_K through Q8_0, plus how to measure on your own workload.
Local AI on AMD and Intel GPUs (September 2026)
What works on AMD Radeon and Intel Arc hardware today, where the official ROCm and SYCL paths stop, and how Vulkan fills the gap.
Running Local LLMs on AMD and Intel GPUs in 2026
AMD's ROCm support splits by card and by OS, Intel archived both of its own LLM libraries this year, and llama.cpp is the path still standing.
What Quantization Actually Costs You in Quality
8-bit measures close to lossless. 4-bit ranges from a 0.6% gain to a 59% drop on one benchmark, depending on the model. The figures, fully attributed.
Local AI Without a GPU: What Actually Runs (October 2026)
Which open models run on CPU and RAM alone, how to size them, why mixture-of-experts helps, and what a machine with no discrete GPU cannot do.
Best Local LLM Runners (September 2026)
Ollama, LM Studio, Jan, llama.cpp and vLLM compared: licences, engines, OpenAI-compatible APIs, multi-user serving, and which to install first.
Local AI by VRAM: Which Models Fit Your GPU (September 2026)
Find your VRAM tier, then the current open-weight pick for chat, coding, vision, speech, translation or agents. Eighteen guides, one index.
Run an LLM Locally on Android and iPhone (September 2026)
Apple caps its on-device model at 4096 tokens per session, and Google's own tables put Gemma 4 E2B at 25.0 decode tokens/sec on an iPhone 17 Pro CPU.
Run LLMs Locally on a Mac: What Actually Fits (September 2026)
Apple Silicon has no discrete VRAM, so tier guides mislead Mac owners. The real ceilings are bandwidth and the GPU-usable slice of unified memory.
GGUF vs AWQ vs GPTQ vs MLX: Which Quantization to Use
Quantization formats compared with real file sizes and bits per weight. Why Q4 does not halve a model, and why low quants generate faster.
MiniMax-M3 Lands in llama.cpp: Sparse Attention and Vision
Two back-to-back merges add MiniMax Sparse Attention and a Qwen2.5-VL style vision tower to llama.cpp, but every existing MiniMax-M3 GGUF must be regenerated.