Ollama
Runs open models on local hardware behind a simple CLI and REST API.
⬦ Free and open source
Released under the MIT license. This page carries no affiliate link. The button goes to the official project.
Get Ollama →Ollama downloads open models and serves them on your own machine. It runs from the command line and exposes a REST API on localhost, which is what coding assistants such as Continue and chat front ends like Open WebUI talk to.
GPU support covers Nvidia cards with compute capability 5.0 and newer, AMD cards through ROCm or Vulkan, and Apple devices through Metal. Which models fit depends on the card: see local AI by VRAM. For a first setup, see the guide to building a local ChatGPT alternative with Ollama and Open WebUI.
The good
- MIT licensed
- Installers for macOS, Windows and Linux, plus an official Docker image
- GPU acceleration on Nvidia, AMD (ROCm or Vulkan) and Apple (Metal)
- Local REST API that editors and chat front ends connect to
The catch
- Also offers models in the cloud, so check where a given model runs if privacy is the point
- Larger models need a GPU with enough VRAM
Pricing
Free and open source to run locally on your own hardware.