Local AI Out of Memory: How to Fix It (November 2026)
A local model that will not load is usually memory arithmetic. Decode the error strings, then cut context, KV cache, parallel requests and layers in order.
Tag
#vram
Mistral Le Chonk: 1 Trillion Parameters, 8-GPU Recipe
Mistral released the 1T open-weight Le Chonk. The official serving recipe assumes 8 GPUs and the smallest GGUF is 233 GB, so the API is the realistic route.
Best Local Chat Models by VRAM Tier (October 2026)
Open-weight chat LLMs by VRAM tier: Qwen3.5, Qwen3.6, Gemma 4 and gpt-oss picks, with Ollama tag sizes and licences.
Why AI Models Need a GPU: A Beginner Explainer (October 2026)
A plain-language walk through parallelism, matrix multiplication, VRAM, memory bandwidth, and why a CPU can run an LLM but rarely fast enough.
Best Local Vision Models by VRAM (October 2026)
Open-weight vision-language models sized by VRAM: Qwen3-VL 2B to 235B-A22B, Mistral Small 3.2, Gemma 3, Pixtral, and where each tier breaks.
Best Local Video Generation Models by VRAM Tier (October 2026)
Open video models sized for real GPUs, with the text encoder counted in and the licence checked. Wan, Hunyuan, LTX-Video, CogVideoX, Mochi and Allegro compared.
MoE vs Dense Models for Local AI (September 2026)
What the 'A3B' and 'A22B' suffixes mean, when Mixture-of-Experts beats a dense model on your hardware, and how to read a model card to tell which is which.
Best Local LLM for Everyday Tasks (Chat, Writing, Summarizing)
Which open-weight LLM should you actually run for everyday chat, writing, and summarizing? A tier-by-tier guide from 8 GB laptops to 32 GB desktops.
Best Local Coding Models by VRAM (September 2026)
Open-weight coding LLMs sized by VRAM: Qwen2.5-Coder, Qwen3-Coder, Devstral Small 2, KAT-Coder, GLM-4.7-Flash, and the trade-offs at each tier.
How much context can a local LLM actually hold? (September 2026)
The advertised context window is one number. What your GPU can actually serve is smaller. The KV cache math, RoPE scaling, and the practical ceiling.
Best Local Reasoning Models by VRAM (October 2026)
Reasoning-capable open-weight models sized by VRAM. DeepSeek-R1, Qwen3 with thinking, QwQ, Phi-4 reasoning, gpt-oss, and the licence traps.
Fine-Tune a Local LLM with LoRA and QLoRA (October 2026)
How to adapt an open-weight model on your own hardware. What LoRA rank and alpha mean, when QLoRA's 4-bit NF4 is worth it, and how to measure it.
Best Used GPUs for Local AI in 2026 (September 2026)
The RTX 30 and 40 series cards that still clear current Ollama and llama.cpp floors, what VRAM tier each opens, and which variants are worth the used premium.
What Quantization Costs You in Quality (October 2026)
How much quality a smaller quant actually loses. Real MMLU and KL Divergence numbers across Q2_K through Q8_0, plus how to measure on your own workload.
How much VRAM does a local LLM actually need? (September 2026)
The published GGUF file size is the weights. VRAM use also includes the KV cache, framework overhead, and your context length. The math, walked through.
What It Takes to Run Qwen3.8-27B on Your Own Hardware
Five stock GGUF quants for Qwen3.8-27B span 7.27 GB to 29.05 GB. A file size is not a VRAM requirement, and a 262,144-token ceiling is not free to fill.
Best GPUs for Local AI (September 2026)
VRAM decides what runs at all, bandwidth decides how fast it types. A buying guide by tier across NVIDIA, AMD, Intel, Apple and the used market.
Best Local Embedding and Reranker Models by VRAM (September 2026)
Local embedding and reranking models for RAG, sized by VRAM. Real GGUF file sizes, licences, and the runtime gap that stops Ollama reranking.
Best Local Image Generation Models by VRAM (2026)
Open image models sized for real GPUs, with the text encoder counted in and the licence checked. FLUX 3, Z-Image, Qwen-Image 2.1 and SD3.5 compared.
Local AI by VRAM: Which Models Fit Your GPU (September 2026)
Find your VRAM tier, then the current open-weight pick for chat, coding, vision, speech, translation or agents. Eighteen guides, one index.
6GB VRAM: What You Can Actually Run Locally (2026)
Local AI on a 6GB GPU: GTX 1660, RTX 2060, RTX 3050 and laptop cards. Real weight sizes for chat, coding, vision, speech, translation and RAG.
GGUF vs AWQ vs GPTQ vs MLX: Which Quantization to Use
Quantization formats compared with real file sizes and bits per weight. Why Q4 does not halve a model, and why low quants generate faster.
12GB VRAM: Every AI Task You Can Run Locally (August 2026)
Local AI on a 12GB GPU: chat, coding, vision, speech and agents for RTX 3060 12GB or RTX 4070. Current picks, per-quant weight sizes, honest limits.
16GB VRAM: Every AI Task You Can Run Locally (August 2026)
Local AI on a 16GB GPU: chat, coding, translation, speech and agents for RTX 4060 Ti, RTX 5060 or Arc A770, and why the vision tier stays unresolved.