Local AI

How to run an LLM locally on Windows (2026)

Run a private, offline LLM on Windows in 2026 - Ollama and LM Studio setup, native vs WSL2, the GPU and driver requirements, and which models fit your VRAM.

Updated July 13, 2026

Running a local LLM on Windows is more straightforward than it used to be: both major tools install natively, with no WSL2 or Docker required, and detect your GPU automatically. The main thing to get right is matching a model to your graphics card’s VRAM. This guide covers the setup and the requirements. For choosing the model itself, see the best local models by VRAM tier, and for the wider case, our run AI locally privacy guide.

The two easiest ways in

Ollama (command line)

Ollama runs as a native Windows service on Windows 10 or later - no WSL2, no Docker. Install it from PowerShell:

irm https://ollama.com/install.ps1 | iex
ollama run gemma4:12b

It detects NVIDIA (CUDA) and AMD (ROCm) GPUs automatically. Per Ollama’s GPU docs, NVIDIA cards need compute capability 5.0 or higher and driver 550 or newer (570+ for the oldest supported cards), and AMD on Windows needs a current ROCm-capable driver stack (Radeon RX 7900 / 9070-series and newer, or Instinct data-centre cards).

LM Studio (graphical)

LM Studio runs on both x64 (which needs AVX2) and ARM (Snapdragon X Elite) Windows. It recommends 16GB of system RAM and at least 4GB of dedicated VRAM, and gives you a model browser and chat interface without the command line.

Native vs WSL2

Ollama and LM Studio both run natively on Windows, and for NVIDIA inference the native path is generally faster than running inside WSL2, which adds a GPU-virtualisation layer. Use WSL2 only if your workflow is already Linux-first.

One exception worth knowing: vLLM, the high-throughput serving engine, has no official native Windows build. The project’s guidance (from community discussion rather than a single official page, so treat it as such) is to run vLLM under WSL2 on Windows. For everyday local chat and coding, you do not need vLLM - Ollama or LM Studio is enough - so this only matters if you are serving a model to multiple users.

Match the model to your GPU

Your VRAM is the ceiling. As a starting point:

  • 4-8GB VRAM (laptop / older cards): gemma4:e4b or gemma4:12b (Google’s Gemma 4 line).
  • 16GB RAM floor, modest GPU: gpt-oss:20b - OpenAI’s open-weight model, which runs on systems with as little as 16GB of memory (about a 14GB download).
  • 24GB VRAM (RTX 3090/4090): qwen3.6:27b (~17GB) or the gemma4:26b mixture-of-experts.

Our VRAM-tier guides go deeper on what fits where.

Quick start

# After installing Ollama:
ollama pull gemma4:12b       # 4-8GB VRAM
ollama pull gpt-oss:20b      # 16GB RAM / modest GPU
ollama pull qwen3.6:27b      # 24GB VRAM

ollama run gemma4:12b

Prefer a graphical app? Install LM Studio, search for one of these models, and download the GGUF build that fits your VRAM. To put a ChatGPT-style web interface on top of Ollama, pair it with Open WebUI.

Everything runs on your own machine - private, free per query, and offline once the model is downloaded.