Local AI

How to Self-Host a Private AI Chat UI

Run a ChatGPT-style chat UI on top of Ollama, LM Studio, or a hosted model API: Open WebUI, LibreChat, AnythingLLM, and the privacy tradeoffs.

A chat UI is the layer most readers picture when they imagine a private ChatGPT: a browser tab, a text box, a streaming reply, a sidebar of past conversations. Running one on your own hardware means the interface, the conversation history, and the prompt travel all live on a computer you control, even if the model itself is served by someone else.

The honest version is more interesting. A self-hosted UI lets you swap the model backend freely: a local Ollama model today, a hosted OpenAI-compatible API tomorrow, a vision model next week. The privacy gain comes from the routing and the storage, not from the UI itself. A chat UI sitting on top of a hosted model is not “private AI”; it is a private interface to someone else’s model. Knowing the difference is the first thing to get right.

1. Choose a UI that fits the job

Three self-hostable chat UIs cover most home setups. They are not clones; they make different tradeoffs, including on licensing: LibreChat and AnythingLLM are MIT licensed, while Open WebUI ships under its own Open WebUI License, a BSD-style licence with an added clause that forbids removing the Open WebUI branding except in deployments with 50 or fewer users in a rolling 30-day period (or under the other exceptions it lists).

Open WebUI is the closest spiritual descendant of ChatGPT for self-hosting. The official documentation describes it as a self-hosted AI platform “built to run entirely offline”, with support for Ollama and OpenAI-compatible APIs. Its GitHub repository lists built-in retrieval-augmented generation, web search, image generation, voice and video calls, and role-based access control with user groups, and names LM Studio, GroqCloud, Mistral, OpenRouter, and vLLM among the OpenAI-compatible endpoints it can point at, alongside several speech-to-text and text-to-speech backends. It is the right default for most home users.

LibreChat is a multi-provider chat UI with a heavier enterprise bent. The LibreChat site lists model selection across Anthropic, AWS, OpenAI, Azure, and other providers, plus single sign-on with OAuth, SAML, and LDAP, and two-factor authentication. It also lists a code interpreter, artifacts, web search, Model Context Protocol support, and persistent memory. Choose LibreChat if you want SSO from day one, or if you want many hosted providers configured side by side with a local backend.

AnythingLLM is less of a chat UI and more of a local-first document and productivity workspace. The AnythingLLM site describes it as MIT licensed and leads with document knowledge, followed by web scraping and search, a local meeting assistant that transcribes and summarises calls, and background jobs; the GitHub README adds scheduled tasks and a no-code agent builder, with multi-user support in the Docker version. Choose AnythingLLM if your main job is chatting with your own PDFs, notes, and call recordings, not holding general conversations.

Pick one for the first install. They overlap heavily, and their default ports collide: Open WebUI’s Docker command below maps port 3000, and LibreChat’s stock docker-compose.yml serves the chat on 3080 but also maps its admin panel to 3000 by default. AnythingLLM’s Docker instructions use 3001. Running two at once means changing a host port first.

2. Install Open WebUI with Docker

Open WebUI is the path most readers should try first. The Open WebUI documentation gives a single Docker command that maps host port 3000 to container port 8080. The version below is the same command with the documentation’s placeholder secret replaced by a randomly generated one:

docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  -e WEBUI_SECRET_KEY=$(openssl rand -hex 32) \
  --name open-webui --restart always \
  ghcr.io/open-webui/open-webui:main

Open http://localhost:3000, create the first account (per the documentation, that first account is the administrator, and sign-up switches itself off once it exists), and the UI is up. WEBUI_SECRET_KEY is the secret used to sign logins; never reuse the placeholder from the documentation. The persistent volume named open-webui keeps chats, users, and uploaded files across container restarts. Note that -p 3000:8080 publishes the port on every network interface of the host, which is Docker’s default; use -p 127.0.0.1:3000:8080 instead to keep the UI reachable only from the machine itself.

If Docker is not available, Open WebUI also installs with pip install open-webui followed by open-webui serve, and then answers on http://localhost:8080, per the same documentation. The README asks for Python 3.11 for this route. The pip install runs directly in your Python environment rather than inside a container, so keep it in a dedicated virtual environment.

3. Connect a local model backend

Open WebUI talks to Ollama over Ollama’s own API on port 11434, and to everything else over the OpenAI-compatible API. Ollama also exposes an OpenAI-compatible endpoint, which other clients can use: Ollama’s OpenAI compatibility documentation gives the local base URL as http://localhost:11434/v1/ and the cloud base URL as ollama.com/v1 over HTTPS. On the local server, the client requires an API key value, but Ollama ignores it.

In Open WebUI, go to Settings > Admin > Connections and, under the Ollama API connections, use host.docker.internal:11434 (with the plain http scheme) as the URL, as the Open WebUI Ollama guide recommends for Docker users with Ollama on the host (the --add-host flag in the Docker command makes that hostname resolve). Pull a model, for example with ollama pull llama3.2, and it appears in the model dropdown.

For a hosted-model connection, add a new OpenAI-compatible connection and paste the OpenAI base URL (https://api.openai.com/v1), your API key, and the model names you want. The chat UI cannot tell the difference at request time. The privacy difference, however, is real: every prompt now leaves your network and is governed by that vendor’s data policy.

4. Network exposure and authentication

A chat UI on localhost:3000 is only useful to the computer it runs on. To reach it from your phone or another machine, you have two reasonable choices: a reverse proxy with HTTPS (Caddy, nginx, or Traefik), or a Tailscale / WireGuard tunnel that keeps the port off the public internet entirely.

Do not skip authentication. A chat UI bound to 0.0.0.0:3000 without login is a public ChatGPT clone that anyone with your IP can use, abuse, or use to spend your API budget. Open WebUI requires an account at first launch, which protects against the casual drive-by, but it is not a substitute for keeping the port off the public internet.

If you do want remote access, put the chat UI behind authentication at the proxy layer (Authentik, oauth2-proxy), keep the chat UI itself on localhost, and add a fail2ban rule for repeated failures. Multi-user installations should also use the roles, groups, and permissions in Open WebUI’s admin panel to limit which models and features each user gets.

5. Keep the privacy promise

Self-hosting the UI is necessary, not sufficient. Three checks decide whether the setup actually keeps prompts private:

  1. Verify the model routing. In Open WebUI’s admin panel, look at the active connection and confirm the API URL is your local Ollama (host.docker.internal:11434) and not a hosted endpoint. A misconfigured connection can route prompts to the cloud without any visible sign in the chat window.
  2. Check telemetry and sharing defaults. Open WebUI’s official Docker image sets SCARF_NO_ANALYTICS=true, DO_NOT_TRACK=true, and ANONYMIZED_TELEMETRY=false in its Dockerfile, and its OpenTelemetry export (ENABLE_OTEL) is off unless you turn it on. Its community-sharing option (ENABLE_COMMUNITY_SHARING) defaults to on; set it to False if you do not want that option offered to users. AnythingLLM is different: its README states that it collects anonymous usage telemetry by default, which you disable by setting DISABLE_TELEMETRY to “true” or in the app under the sidebar’s Privacy section.
  3. Lock down uploads and web search. A chat UI with web search enabled may send your query text to a third-party search provider. RAG with a hosted embedding model sends your documents off-site on first ingest. Read each integration’s data policy and disable what you do not need.

A private chat UI on top of a local model is genuinely private from the model and the UI vendor. A private chat UI on top of a hosted API is private from the UI vendor only. Both are valid setups; they are not the same setup, and naming them accurately is the whole point.

Quick checklist

  • Pick one UI: Open WebUI for general chat, LibreChat for multi-provider enterprise, AnythingLLM for document-heavy work.
  • Install with Docker, set a real WEBUI_SECRET_KEY, and bind to localhost until you need remote access.
  • Connect a local Ollama backend before adding any hosted API key.
  • Verify routing, uploads, and web search settings before pasting sensitive text.
  • Put remote access behind a reverse proxy or a private tunnel; never expose port 3000 directly.