The site’s VRAM tier pages are built for machines with a graphics card. A Raspberry Pi has no graphics card - the Pi 5 ships in 1, 2, 4, 8, or 16 GB variants and the Pi 4 in 1, 2, 4, or 8 GB - and yet it can still load and serve an open-weight model. What it cannot do is match a GPU for speed, and what those RAM tiers will actually run is more constrained than the model cards suggest.
Verified September 2026 against the official Pi spec page, the Ollama and llama.cpp repositories, and the Ollama tag listings.
What a Raspberry Pi Gives You in 2026
raspberrypi.com’s computer page lists every current model. The relevant ones for local AI: the Pi 5 (BCM2712 SoC) in 1, 2, 4, 8, or 16 GB; the Pi 4 Model B (BCM2711) in 1, 2, 4, or 8 GB; the Compute Module 5 in 2, 4, 8, or 16 GB. The Pi Zero 2 W ships with 512 MB and is off the table - more on that later.
A Pi 5 has no dedicated VRAM. Anything on the GPU side is carved from the same LPDDR4X system memory the model uses, so the question is never “how much VRAM”, it is “what fits in system RAM with the OS left enough to keep running”. Raspberry Pi OS Lite on a 2 GB board uses around 200 MB for the kernel plus light services, leaving roughly 1.8 GB for an application. On an 8 GB board that ceiling is closer to 7 GB.
The Engine: llama.cpp on aarch64
llama.cpp - the engine underneath Ollama, LM Studio, and most local-AI tooling - lists KleidiAI as a first-class ARM backend. The build docs name four aarch64 / arm64 targets explicitly: “Linux AArch64 / arm64”, “Android arm64-v8a”, “Apple arm64”, “Windows arm64”. Raspberry Pi OS is 64-bit Debian-derived, so the Linux AArch64 build is the right one. The same page notes that the build flag GGML_CPU_KLEIDIAI=ON is “valid only for AArch64/arm64 builds” and tells older or lower-feature CPUs to “fall back automatically to compatible kernels”.
The llama.cpp install page lists conda-forge, Homebrew, Nix, Winget, and MacPorts as the supported install paths - none of which are the route used on a Pi. In practice Pi users either build llama.cpp from source with cmake -B build && cmake --build build --config Release, run a prebuilt binary, or skip llama.cpp entirely and use Ollama, which packages an aarch64 build that “just works” on Raspberry Pi OS.
What Fits at Each RAM Tier
Sizes below are from the Ollama library’s published tag listings, not from the original repositories. Licences are read per repository from Hugging Face. RAM headroom assumes Raspberry Pi OS Lite plus minimal services, with the OS itself taking under 250 MB.
| RAM on the Pi | Ollama tag | Download size | Context usable | Licence |
|---|---|---|---|---|
| 2 GB | smollm2:135m-instruct-q2_K | 88 MB | Short Q&A only | Apache-2.0 |
| 2 GB | smollm2:360m-instruct-q4_K_M | 271 MB | Short Q&A only | Apache-2.0 |
| 4 GB | tinyllama:1.1b | 638 MB | 1024 tokens | Apache-2.0 |
| 4 GB | qwen3:0.6b | 523 MB | 4096 tokens | Apache-2.0 |
| 8 GB | gemma3:1b | 815 MB | 4096 tokens | gemma (custom, gated) |
| 8 GB | qwen3:1.7b | 1.4 GB | 4096-8192 tokens | Apache-2.0 |
| 16 GB | qwen3:4b | 2.5 GB | 8192 tokens | Apache-2.0 |
Quantization is the lever that decides this table. All the small Ollama tags are 4-bit by default. 2-bit (q2_K) cuts SmolLM2 135M down to 88 MB - usable on a 2 GB Pi but visibly worse at following instructions. 8-bit (q8_0) tags cost roughly twice the 4-bit version - the smollm2:135m-instruct-q8_0 tag is 145 MB, the smollm2:1.7b-instruct-q8_0 tag is 1.8 GB - and exist for when the quality loss is unacceptable.
The Qwen3 README advertises Qwen3 as supporting “100+ languages and dialects” and 256K-token context “extendable up to 1 million tokens” - that is the model, not what fits on a Pi. On a 4 GB board running qwen3:0.6b, holding more than a few thousand tokens of context will swap and stall.
Throughput: What the Pi Can and Cannot Do
This site does not publish tokens-per-second figures for a machine it does not own, and second-hand numbers from a different Pi cooler, power supply, or microSD card class tell you nothing about your own. Two facts are documentable from the primary sources:
- The Pi 5’s CPU - the BCM2712 - is faster per-core than the Pi 4’s BCM2711, but local LLM inference is memory-bandwidth-bound on these parts, so the gap is smaller than clock-speed differences would suggest. Cross-reference the VRAM page - the same bandwidth ceiling drives throughput in both cases.
- The Ollama issue tracker records an “ARM64 performance regression in 0.14.x” closed 26 January 2026 and a request for “Vulkan binary releases for Raspberry Pi OS / Pi 4” closed 4 April 2026, alongside an open issue that “Vulkan backend ignoring
maxComputeWorkGroupInvocations” “breaks Pi 5 / V3D”. Ollama on a Pi is officially supported enough to ship a Pi-specific bug history, not officially blessed with a hardware-tuning guide.
Quantize aggressively before you give up. Dropping from a 2.5 GB qwen3:4b to a 1.4 GB qwen3:1.7b is a larger throughput win than any setting Ollama exposes at runtime.
What Does Not Work on a Raspberry Pi
Pi Zero 2 W and the 512 MB SKUs. 512 MB is below llama.cpp’s working set on every published model - the smallest, smollm2:135m-instruct-q2_K at 88 MB, will load, but the context allocation plus the OS will swap itself to death. Treat these as excluded.
Older Pi models without a 64-bit path. Every Pi from the 3 Model B forward boots a 64-bit Raspberry Pi OS. The original Pi and Pi Zero do not, and the 32-bit builds are not what the published Ollama and llama.cpp binaries target. Same practical answer: excluded.
Long contexts on small Pi boards. Holding the full system prompt plus a long chat history is the workload CPU inference pays for first. Short Q&A and one-shot generation are the shapes the Pi handles best; Retrieval-Augmented Generation across a folder of PDFs is the shape that hurts most.
Anything that wants the GPU. Ollama’s GPU documentation does not list the Pi’s VideoCore as an option. The closed issue “Vulkan backend ignoring maxComputeWorkGroupInvocations (breaks Pi 5 / V3D)” on the Ollama issue tracker is the project’s own admission that Vulkan on the Pi 5 is not stable today. Treat the Pi as a CPU-only machine for inference purposes.
What to Decide Before You Start
Pick the model family with the licence in mind. Apache-2.0 models - Qwen3, SmolLM2, TinyLlama - have no usage clause beyond attribution. Gemma 3 is custom-licensed and the Hugging Face repository is gated (the Ollama tag downloads without prompting). Any consumer reading the open-model licences hub before a commercial build is doing it the right way around.
Use a fast microSD card, or boot from USB SSD. Inference is memory-bandwidth-bound, and a slow card turns every load-time into a minute-long wait. Class A1/A2 microSD cards or a USB 3 SSD on a Pi 4 / Pi 5 with the USB boot enabled eliminates most of that.
Plan for thermals. The Pi 5 runs hot under sustained CPU load; the official Active Cooler is not optional for continuous inference. The Pi 4 throttles slower but is not immune.
And when the Pi stops being enough, the answer is not “wait for a faster Pi”. The without-a-GPU guide covers the broader CPU-only path on a desktop or laptop with 16 to 64 GB of system memory, where the Pi-class constraint of choosing between Qwen 0.6B and SmolLM 135M stops applying. If you do have a graphics card to spend, the VRAM-by-tier hub is the right entry.
Bottom Line
A Raspberry Pi 5 with 4 GB or 8 GB of RAM is a real local-inference machine, and a Pi 4 with 4 GB is usable in a pinch. The smallest Ollama tags - smollm2:135m-instruct-q2_K at 88 MB, qwen3:0.6b at 523 MB, tinyllama:1.1b at 638 MB - load on every 2 GB-or-larger Pi and respond to short questions. qwen3:1.7b at 1.4 GB on an 8 GB board is the case where a Pi actually feels useful rather than demo-grade.
The catches are the ones this page has named rather than the ones a benchmark chart would show: 16 GB models do not fit on any Pi this generation ships, Vulkan on the Pi 5 GPU is documented by Ollama’s own issues as unstable, long contexts are the workload the Pi pays first, and the only throughput number worth trusting is the one you measure on the board in front of you.