Running an LLM entirely inside the browser is no longer a demo. WebGPU is now enabled by default in Chrome 113 and later, in Edge, in Safari 26 on iOS, and in Safari 26 on macOS in partial form, while Firefox turns it on by default on Windows and on Apple Silicon Macs but not yet on Linux, Intel Macs or Android (caniuse.com WebGPU table, Firefox 147 release notes, MDN compatibility data). Two mature open-source libraries sit on top: Hugging Face’s Transformers.js (default backend WASM, optional WebGPU via device: 'webgpu') and WebLLM (WebGPU-only, ships its own engine). Both keep every prompt and every weight on the reader’s device, which is the actual reason this category exists at all.
Verified October 2026 against the live caniuse support table, the Transformers.js docs and v4.3.1 / v4.3.0 / v4.2.0 release notes, the WebLLM README and prebuilt-model config, the Hugging Face WebGPU-kernels post, the Chrome developer docs, the MDN WebGPU reference, and (for Firefox, re-checked 2026-10-11) the Firefox 147 release notes and MDN’s compatibility data.
What “in the browser” actually means
The browser is just the shell. Underneath, a JavaScript library loads model weights from the Hugging Face Hub (or from a same-origin URL you control), parses them, and dispatches operations to one of two backends.
- WASM (WebAssembly). CPU-only, runs everywhere WebAssembly does, default in Transformers.js. The dtypes guide lists the typical data types as
q8on WASM,fp32on WebGPU, withq4,q8,fp16andfp32all selectable per model (Transformers.js dtypes guide). - WebGPU. General-purpose GPU compute, exposed as a web standard. MDN describes WebGPU as the “successor to WebGL” with “better compatibility with modern GPUs” and “first-class support for GPGPU computations” (MDN WebGPU API). For inference, that translates into roughly an order of magnitude throughput over WASM at the same model size, with the catch that the user’s browser, OS, GPU and driver have to all line up.
The September 2026 story on the kernel side still holds on the numbers: the Hugging Face WebGPU kernels release shipped 207 operation-level kernels (“matrix multiplications, normalizations, convolutions, attention primitives, quantization operations, data-layout transformations”) published as versioned packages at huggingface.co/webgpu-kernels. The release post measures 2.57x faster by geometric mean and 1.90x faster at the median than ORT WebGPU on an Apple M4 GPU, with one case (a bilinear Einsum) running more than 10,000x faster. The kernels are operations, not models - they sit below higher-level runtimes. The catch is the same one Hugging Face flags directly: “Two shaders can implement the same operation and produce the same output while behaving completely differently across different accelerators.”
Browser support and the remaining Firefox gaps
The caniuse WebGPU table is the usual reference. As of October 2026, global full support is around 85.7% of users, with another 3.1% partial. Chrome and Edge support it natively from version 113 onward (Chrome is currently at 154, Edge at 154), as does Samsung Internet from 24 onward (currently at 30) and Opera from 99 (currently at 135). Safari is split: iOS Safari supports WebGPU from version 26 (currently at 27.0), while desktop Safari has the same floor with the partial-support marker attached, which caniuse explains as WebGPU being enabled by default only on macOS 26 Tahoe or later (currently at 27). Firefox support is partial and depends on the platform (Firefox is currently at 157). Mozilla’s compatibility data on MDN lists WebGPU on Windows from Firefox 141 and on macOS Tahoe on Apple Silicon from 145, and the Firefox 147 release notes extend it to “devices with Apple Silicon processors on all supported macOS versions” (caniuse still describes the macOS default as Tahoe-only). The same MDN data still lists no support on Linux, on Intel Macs, in service workers, or in Firefox for Android; on desktop, caniuse notes it can be enabled with the dom.webgpu.enabled flag. The Transformers.js docs flag the same asymmetry in their warning (“The WebGPU API is still experimental in many browsers”), and document the three flag routes for non-defaulted browsers (Transformers.js WebGPU guide). Transformers.js itself added explicit WebGPU on Safari 26+ in its v4.3.0 release (2026-09-16), but the underlying Safari browser-side partial-support marker is what the caniuse row is reflecting.
The practical consequence is that a WebGPU-only product reaches roughly six out of seven readers and silently fails on the seventh, mostly Firefox on Linux, Intel Macs and Android, plus older browsers. Any reader-facing feature that depends on WebGPU should either feature-detect ("gpu" in navigator), fall back to WASM at lower speed, or block. None of these are free to implement, which is why no demo that “just works” is one script line in practice.
What models actually fit
WebLLM’s prebuilt-model config is the cleanest source for in-browser LLM sizes. The repo’s src/config.ts declares vram_required_MB for every prebuilt model, and the file size runs in proportion. The smallest chat-model entry is SmolLM2-135M-Instruct at 359.69 MB (an embedding model, snowflake-arctic-embed-s, is smaller at 238.71 MB), the Llama-3.1-8B-Instruct and DeepSeek-R1-Distill-Llama-8B q4f32 builds sit at 6,101.01 MB each, and the largest prebuilts are the Llama-3-70B and Llama-3.1-70B q3f16 builds at 31,153.13 MB each. Transformers.js adds new architectures every minor release; v4.2.0 (released 2026-04-23) added tool support to TextGenerationPipeline, v4.3.0 (released 2026-09-16) added structured output, three new model architectures (DeepSeek-V4, Zaya, HRM-Text), and WebGPU support for Safari 26 and above, and v4.3.1 (released 2026-10-07) added EmbeddingGemma 2 support. The v4.0 announcement described running GPT-OSS 20B (q4f16) at roughly 60 tokens per second on an M4 Pro Max (transformers.js releases).
| Family | Model (WebLLM prebuilt) | VRAM (MB) | Realistic browser device |
|---|---|---|---|
| SmolLM2 | SmolLM2-135M-Instruct-q0f16 | 360 | Any laptop, phone |
| SmolLM2 | SmolLM2-360M-Instruct-q4f16_1 | 376 | Any laptop, phone |
| Llama 3.2 | Llama-3.2-1B-Instruct-q4f16_1 | 879 | Any modern laptop |
| Llama 3.2 | Llama-3.2-3B-Instruct-q4f16_1 | 2,264 | 8 GB+ device |
| Phi-4 | Phi-4-mini-instruct-q4f16_1 | 3,438 | Discrete GPU or M-series |
| Mistral | Mistral-7B-Instruct-v0.3-q4f16_1 | 4,573 | 8 GB+ discrete GPU |
| Llama 3.1 | Llama-3.1-8B-Instruct-q4f32_1 | 6,101 | 8 GB+ discrete GPU |
VRAM figures from the WebLLM prebuilt config. These are the weights-plus-activation working set, not the on-disk download size, and they assume the default 4,096-token context; longer contexts grow the figure roughly linearly with the KV cache. The arithmetic and the tradeoffs between q4 and q8 are the same as on a desktop: the quantization formats page covers the format-level differences and the quality-cost page covers what each level costs in output quality.
Transformers.js vs WebLLM
The two libraries overlap in goal and diverge in scope. Both are Apache 2.0-licensed, both run in a browser, both expose OpenAI-compatible chat APIs. The differences that matter for a reader picking one are in the table below.
| Transformers.js | WebLLM | |
|---|---|---|
| Default backend | WASM (CPU) | WebGPU only |
| Engine | ONNX Runtime Web | TVMjs with MLC-LLM |
| Model format | ONNX (any Hub model converted via Optimum) | MLC prebuilt (curated list) |
| Fallback path | Yes (WASM if WebGPU absent) | None - WebGPU required |
| Latest release | v4.3.1, 2026-10-07 (releases) | v0.2.86, 2026-10-11 (releases) |
| OpenAI-compatible API | Yes (chat, embeddings, etc., with libraries like @huggingface/transformers wrappers) | Yes (/v1/chat/completions family) |
| WebGPU risk surface | One device: 'webgpu' flag (docs) | None, the runtime assumes WebGPU |
A reader who wants a quick demo that works in Firefox falls back to Transformers.js with the default WASM backend; a reader who wants the fastest possible browser LLM on Chrome picks WebLLM with a prebuilt model. Both end up at the same runner-software comparison decision as on a desktop, just with the browser pinned as the host.
The hidden cost of “free”
In-browser inference is not actually free. Three costs show up that no demo page mentions.
Bandwidth is a power bill. The model has to download once before the first token. A 6 GB Llama-3.1-8B is six gigabytes of cellular data on a phone or a slow cafe Wi-Fi. WebLLM caches weights in the Cache API or OPFS (WebLLM README) so subsequent loads are local, but the first load is a real cost.
First-token latency is a UX cost. Compilation of WebGPU shaders on first run is in the seconds-to-tens-of-seconds range; Transformers.js documents the same kind of warmup for its WebGPU runtime (Transformers.js WebGPU guide). A reader who loads a tab and waits 30 seconds for a reply is a reader who closes the tab.
Privacy is not absolute. The model never leaves the device, and the prompt never leaves the device, but the model file itself usually comes from a third-party CDN. A reader who needs the weights and the prompt to stay inside a network boundary has to self-host both. That is the same problem as running locally on a no-GPU box, with the same answer: point the runtime at a URL you control.
Bottom line
WebGPU-based browser inference is real, Apache 2.0-licensed, and works today in Chrome, Edge, Opera, Samsung Internet and Safari on iOS. Firefox is the main remaining gap among current browsers: it turns WebGPU on by default on Windows and on Apple Silicon Macs, but not yet on Linux, Intel Macs or Android. Transformers.js (WASM by default, WebGPU optional) is the safer pick when you need a fallback; WebLLM (WebGPU-only) is the faster pick when you know the browser supports it. The Hugging Face WebGPU-kernels release is the next step down in the stack - operation-level primitives that higher-level runtimes can build on - and is a watch, not a thing to install yet. Pick a model whose VRAM footprint fits the device you are actually targeting, expect a long first load, and feature-detect WebGPU rather than assuming it.