If you want to run an open-weight model on your own hardware, four backends dominate the conversation in 2026: Ollama, LM Studio, the llama.cpp server, and vLLM. They all expose an OpenAI-compatible HTTP API, they all run on a single workstation, and they all happily answer a curl request. They differ enough, though, that the choice between them shapes what hardware you buy, which model formats you can use, and how much tuning you do by hand.
Key Takeaways
- Ollama is the easiest “it just works” path: install on macOS, Linux, Windows, or Docker, pull a model with
ollama pull llama3.2, and the OpenAI-compatible endpoint is already running onhttp://localhost:11434/v1. - LM Studio is the most polished desktop GUI, ships with an in-app model browser, and exposes
http://localhost:1234/v1once you toggle “Start server” in the Developer tab. - llama.cpp server (
llama-server) is the lowest-level choice: a single C++ binary that loads GGUF files, supports the widest set of GPU backends, and has the smallest surface area to debug. - vLLM is the production-grade engine built for throughput, continuous batching, and 200-plus Hugging Face architectures, but it expects a real NVIDIA or AMD GPU and is overkill for a single chat user.
The Model-Format Split
The single biggest difference between these four is what they will load.
| Backend | Model format | How you add a model |
|---|---|---|
| Ollama | GGUF (also pulls from the Ollama library) | ollama pull <name> or import a GGUF |
| LM Studio | GGUF only (.gguf files) | Drag-and-drop or in-app Discover tab |
| llama.cpp server | GGUF only | Point -m at a .gguf on disk |
| vLLM | Hugging Face transformers (safetensors) | Point at a HF repo: vllm serve meta-llama/Llama-3.1-8B-Instruct |
The Ollama library exposes hundreds of pre-quantised GGUF builds, and LM Studio adds a Discover tab that searches Hugging Face for GGUF files you can download with a click. llama.cpp server will only load GGUF files you point it at. vLLM inverts the assumption and reads safetensors straight from a Hugging Face repo, with quantisation handled at load time via AWQ, GPTQ, FP8, INT8, INT4, and several other formats rather than at download time. For a reader who wants to run an unquantised or FP8 build straight from Hugging Face, vLLM is the path of least friction; for everyone else, the GGUF ecosystem is deeper and more uniform.
The OpenAI-Compatible API Surface
All four expose an OpenAI-compatible HTTP endpoint, but the paths, ports, and feature coverage differ.
| Backend | Default port | Base URL | Notes |
|---|---|---|---|
| Ollama | 11434 | http://localhost:11434/v1 | Native API at /api/chat plus OpenAI compat at /v1/chat/completions, /v1/completions, /v1/embeddings, /v1/models, and /v1/responses (added in v0.13.3) |
| LM Studio | 1234 | http://localhost:1234/v1 | Toggle “Start server” in Developer tab or run lms server start --port 1234 |
| llama.cpp server | 8080 | http://localhost:8080/v1 | OpenAI compat plus extra endpoints (/completion, /infill, /lora-adapters, /props) |
| vLLM | 8000 | http://localhost:8000/v1 | OpenAI compat plus Anthropic Messages API and gRPC |
The practical consequence is that any tool built against the OpenAI Python or JS SDK will work against all four by changing the base_url. The Ollama OpenAI-compatibility layer, for example, accepts model, messages, temperature, top_p, max_tokens, tools, response_format, stream, seed, stop, and reasoning_effort, but rejects logit_bias, n, user, and tool_choice. That is close enough to OpenAI for almost every client, and Ollama’s /v1/responses endpoint adds OpenAI’s newer Responses API on top of the legacy chat completions endpoint. The llama.cpp server lists the same set of OpenAI-compatible endpoints plus a real-time /v1/chat/completions/control route for reasoning_end, which is useful for chains of tool calls.
Defaults That Matter When You Stack Things
Concurrency and keep-alive defaults tell you what each server assumes about your workload.
- Ollama binds
127.0.0.1:11434by default, setsOLLAMA_NUM_PARALLEL=1(max parallel requests per model),OLLAMA_MAX_LOADED_MODELSto 3 times the number of GPUs (3 for CPU-only), andOLLAMA_KEEP_ALIVE=5mso models unload from memory five minutes after the last request. - LM Studio uses a built-in chat UI plus the developer toggle and defaults to port 1234; concurrency is governed by the UI’s per-model load state.
- llama.cpp server defaults to
--host 127.0.0.1,--port 8080,--parallel -1(auto), with continuous batching enabled by default. Setting--api-keyadds a bearer-token requirement on top of the OpenAI endpoints. - vLLM treats every request as part of a continuous batch and uses PagedAttention to manage KV cache, so concurrency is the design point, not a constraint.
The Ollama defaults are tuned for one user at a time on a workstation, which is exactly what most readers want. The llama.cpp server’s --parallel -1 (auto) lets the same binary scale up by reading available memory at startup, and vLLM is built for many simultaneous users from the ground up.
Hardware Support
| Backend | NVIDIA CUDA | Apple Silicon / Metal | AMD ROCm / HIP | Intel / SYCL | Vulkan | CPU only |
|---|---|---|---|---|---|---|
| Ollama | Yes | Yes | Yes | Yes | Yes | Yes |
| LM Studio | Yes (Windows/Linux) | Yes (macOS) | Limited | No | No | Yes |
| llama.cpp server | Yes | Yes | Yes | Yes | Yes | Yes |
| vLLM | Yes (primary) | Limited (CPU/Metal experimental) | Yes | Limited (Gaudi plugin) | No | Yes |
LM Studio’s system requirements page lists Apple Silicon (M1, M2, M3, M4) as the supported macOS path, with macOS 14.0 or newer and 16 GB of RAM recommended, plus Windows x64 and ARM64 (Snapdragon X Elite), and Linux x64 and ARM64 via AppImage. AVX2 is required on x64. Ollama is broader: it ships a single binary that runs on Linux, macOS, Windows, and inside Docker, and the Ollama FAQ explicitly walks through setting OLLAMA_HOST=0.0.0.0 to expose the server on a LAN. llama.cpp server has the broadest GPU-backend support of any of these, with CUDA, Metal, ROCm/HIP, SYCL, Vulkan, and plain CPU all in the build matrix. vLLM is the most demanding: NVIDIA CUDA is the primary target, AMD ROCm is supported, and Apple Silicon CPU inference works but the Metal path is still marked experimental in the docs.
Which One Should You Pick
There is no single right answer, but the trade-offs sharpen quickly once you describe your workload.
- Picking Ollama. If you want a single command (
ollama run llama3.2), a clean Docker image for a home server, an OpenAI-compatible API onlocalhost:11434, and a managed library of pre-quantised models, Ollama is the path of least resistance. The community is large enough that almost every “how do I run X locally” tutorial assumes you are running Ollama. - Picking LM Studio. If you want a GUI, an in-app Discover tab for searching Hugging Face for GGUF files, and a one-click server toggle without touching a terminal, LM Studio is the most polished experience on macOS or Windows. The CLI (
lms server start --port 1234) is fine, but the GUI is the differentiator. - Picking llama.cpp server. If you want maximum control over a single model on unusual hardware (a Raspberry Pi, an AMD RDNA card, an Intel Arc GPU, a Vulkan device), the llama.cpp server is the lowest-friction option. It loads GGUF files straight from disk, exposes every flag Ollama hides, and supports the broadest set of GPU backends.
- Picking vLLM. If you want to serve a model to many users at once, you have a real NVIDIA GPU, and you care about tokens-per-second-per-GPU more than you care about how the model got onto the box, vLLM is the production-grade engine. Continuous batching and PagedAttention are not features you can fake with a config flag.
For most readers of this site, the practical answer is Ollama or LM Studio on a workstation with a single GPU, with llama.cpp server as the fallback when hardware is unusual and vLLM only when the workload has outgrown a single user.
What This Means
The four-backends question looks complicated from the outside and collapses to a single axis once you describe your workload: how many users, what hardware, and whether you want a GUI. Ollama, LM Studio, and llama.cpp server all target the one-user-on-a-workstation case and trade features against polish. vLLM targets the many-users-on-a-GPU case and trades portability for throughput. If you are not sure which one to pick, start with Ollama; if you hit a wall on hardware, switch to llama.cpp server; if you need throughput, switch to vLLM. The OpenAI-compatible API is the constant that makes the switches cheap.
The Bottom Line
Ollama, LM Studio, llama.cpp server, and vLLM are four answers to the same question, not four different questions. They all expose an OpenAI-compatible HTTP API, they all run on commodity hardware, and they all happily load a Qwen, Llama, Gemma, or gpt-oss model. Ollama is the easiest install; LM Studio is the nicest UI; llama.cpp server is the most flexible on hardware; vLLM is the fastest under load. Pick the one that matches the workload and keep the others in your back pocket, because the OpenAI-compatible API is what makes switching cheap.