Running an LLM entirely inside the browser is no longer a demo. WebGPU is now enabled by default in Chrome 113 and later, in Edge, in Safari 26 on iOS, and in Safari 26 on macOS in partial form, and Firefox remains the gap (caniuse.com WebGPU table). Two mature open-source libraries sit on top: Hugging Face’s Transformers.js (default backend WASM, optional WebGPU via device: 'webgpu') and WebLLM (WebGPU-only, ships its own engine). Both keep every prompt and every weight on the reader’s device, which is the actual reason this category exists at all.
Verified September 2026 against the live caniuse support table, the Transformers.js docs and v4.2.0 release notes, the WebLLM README and prebuilt-model config, the Hugging Face WebGPU-kernels post, the Chrome developer docs, and the MDN WebGPU reference.
What “in the browser” actually means
The browser is just the shell. Underneath, a JavaScript library loads model weights from the Hugging Face Hub (or from a same-origin URL you control), parses them, and dispatches operations to one of two backends.
- WASM (WebAssembly). CPU-only, runs everywhere WebAssembly does, default in Transformers.js. The dtypes guide lists the typical data types as
q8on WASM,fp32on WebGPU, withq4,q8,fp16andfp32all selectable per model (Transformers.js dtypes guide). - WebGPU. General-purpose GPU compute, exposed as a web standard. MDN describes WebGPU as the “successor to WebGL” with “better compatibility with modern GPUs” and “first-class support for GPGPU computations” (MDN WebGPU API). For inference, that translates into roughly an order of magnitude throughput over WASM at the same model size, with the catch that the user’s browser, OS, GPU and driver have to all line up.
The September 2026 story on the kernel side is the Hugging Face WebGPU kernels release, which shipped 207 operation-level kernels (“matrix multiplications, normalizations, convolutions, attention primitives, quantization operations, data-layout transformations”) published as versioned packages at huggingface.co/webgpu-kernels. The release post measures 2.57x faster by geometric mean and 1.90x faster at the median than ORT WebGPU on an Apple M4 GPU, with one case (a bilinear Einsum) running more than 10,000x faster. The kernels are operations, not models - they sit below higher-level runtimes. The catch is the same one Hugging Face flags directly: “Two shaders can implement the same operation and produce the same output while behaving completely differently across different accelerators.”
Browser support and the Firefox gap
The caniuse WebGPU table is the canonical answer. As of September 2026, global support is around 85.5% of users. Chrome and Edge support it natively from version 113 onward (Chrome is currently at 151), as does Samsung Internet from 24 onward and Opera from 99. Safari is split: iOS Safari supports WebGPU from version 26, while desktop Safari has the same floor with the partial-support marker attached, meaning some operations work and some do not. Firefox is the entire gap: it is currently at version 157 and the feature remains disabled by default behind dom.webgpu.enabled. Mozilla has not announced a default-on date. The Transformers.js docs flag the same asymmetry in their warning (“The WebGPU API is still experimental in many browsers”), and document the three flag routes for non-defaulted browsers (Transformers.js WebGPU guide).
The practical consequence is that a WebGPU-only product reaches roughly six out of seven readers and silently fails on the seventh, the Firefox share. Any reader-facing feature that depends on WebGPU should either feature-detect ("gpu" in navigator), fall back to WASM at lower speed, or block. None of these are free to implement, which is why no demo that “just works” is one script line in practice.
What models actually fit
WebLLM’s prebuilt-model config is the cleanest source for in-browser LLM sizes. The repo’s src/config.ts declares vram_required_MB for every prebuilt model, and the file size runs in proportion. The smallest entry is SmolLM2-135M-Instruct at 359.69 MB and the largest prebuilts are the Llama-3.1-8B-Instruct and DeepSeek-R1-Distill-Llama-8B builds at 6,101.01 MB each. Transformers.js adds new architectures every minor release; v4.2.0 (released 2026-04-23) added tool support to TextGenerationPipeline and the v4.0 announcement described running GPT-OSS 20B (q4f16) at roughly 60 tokens per second on an M4 Pro Max (transformers.js releases).
| Family | Model (WebLLM prebuilt) | VRAM (MB) | Realistic browser device |
|---|---|---|---|
| SmolLM2 | SmolLM2-135M-Instruct-q0f16 | 360 | Any laptop, phone |
| SmolLM2 | SmolLM2-360M-Instruct-q4f16_1 | 376 | Any laptop, phone |
| Llama 3.2 | Llama-3.2-1B-Instruct-q4f16_1 | 879 | Any modern laptop |
| Llama 3.2 | Llama-3.2-3B-Instruct-q4f16_1 | 2,264 | 8 GB+ device |
| Phi-4 | Phi-4-mini-instruct-q4f16_1 | 3,438 | Discrete GPU or M-series |
| Mistral | Mistral-7B-Instruct-v0.3-q4f16_1 | 4,573 | 8 GB+ discrete GPU |
| Llama 3.1 | Llama-3.1-8B-Instruct-q4f32_1 | 6,101 | 8 GB+ discrete GPU |
VRAM figures from the WebLLM prebuilt config. These are the weights-plus-activation working set, not the on-disk download size, and they assume the default 4,096-token context; longer contexts grow the figure roughly linearly with the KV cache. The arithmetic and the tradeoffs between q4 and q8 are the same as on a desktop: the quantization formats page covers the format-level differences and the quality-cost page covers what each level costs in output quality.
Transformers.js vs WebLLM
The two libraries overlap in goal and diverge in scope. Both are Apache 2.0-licensed, both run in a browser, both expose OpenAI-compatible chat APIs. The differences that matter for a reader picking one are in the table below.
| Transformers.js | WebLLM | |
|---|---|---|
| Default backend | WASM (CPU) | WebGPU only |
| Engine | ONNX Runtime Web | TVMjs with MLC-LLM |
| Model format | ONNX (any Hub model converted via Optimum) | MLC prebuilt (curated list) |
| Fallback path | Yes (WASM if WebGPU absent) | None - WebGPU required |
| Latest release | v4.2.0, 2026-04-23 (releases) | v0.2.83, 2026-04-24 (releases) |
| OpenAI-compatible API | Yes (chat, embeddings, etc., with libraries like @huggingface/transformers wrappers) | Yes (/v1/chat/completions family) |
| WebGPU risk surface | One device: 'webgpu' flag (docs) | None, the runtime assumes WebGPU |
A reader who wants a quick demo that works in Firefox falls back to Transformers.js with the default WASM backend; a reader who wants the fastest possible browser LLM on Chrome picks WebLLM with a prebuilt model. Both end up at the same runner-software comparison decision as on a desktop, just with the browser pinned as the host.
The hidden cost of “free”
In-browser inference is not actually free. Three costs show up that no demo page mentions.
Bandwidth is a power bill. The model has to download once before the first token. A 6 GB Llama-3.1-8B is six gigabytes of cellular data on a phone or a slow cafe Wi-Fi. WebLLM caches weights in the Cache API or OPFS (WebLLM README) so subsequent loads are local, but the first load is a real cost.
First-token latency is a UX cost. Compilation of WebGPU shaders on first run is in the seconds-to-tens-of-seconds range; Transformers.js documents the same kind of warmup for its WebGPU runtime (Transformers.js WebGPU guide). A reader who loads a tab and waits 30 seconds for a reply is a reader who closes the tab.
Privacy is not absolute. The model never leaves the device, and the prompt never leaves the device, but the model file itself usually comes from a third-party CDN. A reader who needs the weights and the prompt to stay inside a network boundary has to self-host both. That is the same problem as running locally on a no-GPU box, with the same answer: point the runtime at a URL you control.
Bottom line
WebGPU-based browser inference is real, Apache 2.0-licensed, and works today in Chrome, Edge, Opera, Samsung Internet and Safari on iOS. The Firefox share is the single biggest gap and there is no announced default-on date. Transformers.js (WASM by default, WebGPU optional) is the safer pick when you need a fallback; WebLLM (WebGPU-only) is the faster pick when you know the browser supports it. The Hugging Face WebGPU-kernels release is the next step down in the stack - operation-level primitives that higher-level runtimes can build on - and is a watch, not a thing to install yet. Pick a model whose VRAM footprint fits the device you are actually targeting, expect a long first load, and feature-detect WebGPU rather than assuming it.