Best vector databases for self-hosted RAG (October 2026)

Six self-hosted vector databases for RAG compared: Qdrant, Milvus, Chroma, Weaviate, pgvector, LanceDB. Licence, indexing, the catch.

Updated November 1, 2026

Six self-hosted vector databases will store the embeddings for a local RAG stack this afternoon. The cluster already covers the orchestration layer in the RAG setups guide and the embedder and reranker model choices in the by-VRAM embedding guide. This page covers what sits underneath both: the database that holds the vectors and runs the nearest-neighbour search at query time. Picking it is the first decision that locks in how your stack scales, how it fails, and what you have to rebuild when you change embedders.

The comparison

DatabaseLicenceLanguageLatest verified releaseDeployment modesIndexingSDKs
QdrantApache 2.0 (Qdrant Solutions GmbH, 2026)Rustv1.20.0 (2026-10-22)Client-server; Qdrant Edge embedded variantHNSW, payload indexes, ACORN filterPython, JS/TS, Rust, Go, .NET, Java
MilvusApache 2.0Go and C++v3.0.5 (2026-10-30)Lite (embedded Python), Standalone (Docker), Distributed (Kubernetes)HNSW, IVF, FLAT, SCANN, DiskANN, CAGRA GPU indexPython (PyMilvus), Go, Java, Node.js, C++, C# (Microsoft-contributed)
ChromaApache 2.0Rust1.5.9 (2026-05-05); the latest pre-release tag ships 1.5.10.dev326In-memory, persistent disk, client-server; Chroma Cloud (hosted, serverless)HNSWPython, JS/TS
WeaviateBSD-3-Clause for code outside wl/; a separate Weaviate enterprise licence for code inside wl/ (Weaviate BV, 2026-present)Gov1.40.3 (2026-10-28)Embedded (experimental), Docker, Kubernetes, Weaviate CloudHNSW, Flat, Dynamic, HFresh (HFresh only supports cosine and l2-squared)Python, JS/TS, Java, Go, C#/.NET
pgvectorPostgreSQL License (BSD-style)Cv0.8.7A PostgreSQL extension; no separate serverHNSW, IVFFlat; vector (2,000 dim indexed), halfvec (4,000), bit (64,000), sparsevec (1,000 non-zero)Any language with a Postgres client; the README lists libraries or examples for 35 language rows
LanceDBApache 2.0Rust core, multi-language wrappersv0.41.0 (2026-10-29)Embedded (library, runs in-process); LanceDB Enterprise (distributed)IVF-PQ, IVF-HNSW; GPU index buildingPython, JS/TS, Java, Rust

All six are Apache-2.0 or BSD-family permissive except Weaviate, whose weaviate/ repository is split: the BSD-3-Clause licence applies to the OSS tree, and code under the wl/ directory is “only available under a separate enterprise license”, with enterprise features enabled by licence key. Readers who care about redistribution should read the Weaviate LICENSE file directly before packaging the binary.

Picking by what you already have

The cheapest pick is the database that piggy-backs on infrastructure you are already running. Each entry below is a one-line answer; the next section covers when those answers are wrong.

You already run PostgreSQL. pgvector. v0.8.7 ships with HNSW and IVFFlat indexes, four vector types (vector, halfvec, bit, sparsevec), six distance metrics (L2, inner product, cosine, L1, Hamming, Jaccard), and full Postgres features (ACID, point-in-time recovery, JOINs), plus hybrid search with Postgres full-text search and binary quantization. Everything is in the same database as your application data, so embedding metadata, user permissions, and backups already work. Cost: vector indexes still eat RAM; a million 768-dim float32 vectors are 3.07 GB (2.86 GiB) of raw data before the HNSW graph and the table heap.

You write Python and want zero moving parts. LanceDB (embedded) or Milvus Lite. Both run inside your Python process: import lancedb and the database file lives next to your script; pymilvus in Lite mode is the same idea with a different on-disk format. LanceDB documents IVF-PQ and IVF-HNSW indexes plus GPU index building; the current Milvus Lite documentation lists FLAT, BRUTE_FORCE, IVF_FLAT, IVF_SQ8, HNSW, HNSW_SQ and AUTOINDEX.

You want a single server you can drop on a host. Qdrant. One Docker image started with docker run -p 6333:6333 qdrant/qdrant; the HNSW index, payload indexes, and the ACORN search algorithm (which Qdrant describes as improving accuracy with multiple high-cardinality filters) are all in the box. It is written in Rust, and the official SDK list (Python, JS/TS, Rust, Go, .NET, Java) covers the usual stacks.

You want Kubernetes and billion-scale from day one. Milvus Distributed. v3.0.5 (released 2026-10-30) is the current line; the docs list Lite, Standalone (single Docker image), and Distributed (Kubernetes, cloud-native). Indexes cover the full menu: HNSW, IVF, FLAT, SCANN, DiskANN, and the NVIDIA CAGRA GPU index. Weaviate on Kubernetes, below, is the other option at this scale.

You want vectorizers baked in. Weaviate. Its model-provider integrations (Cohere, OpenAI, Hugging Face, Voyage AI, and others) can generate embeddings at insert and search time, so the application code only sends text. v1.40.3 (released 2026-10-28) ships HNSW, Flat, Dynamic and HFresh; HFresh is a disk-based, cluster-based index, and it only supports cosine and l2-squared distance. The trade is the licence split mentioned above: code outside wl/ is BSD-3, code inside it is under a separate enterprise licence.

You want a managed option with a self-hosted fallback. Weaviate Cloud or Chroma Cloud. Both have self-hosted paths and hosted paths. Chroma’s documentation covers in-memory, persistent and client-server modes for self-hosted runs, and single-node Chroma collections use an HNSW index.

Picking by scale

The same database behaves very differently across the three bands a local stack actually hits.

One user, one laptop, tens of thousands of chunks. LanceDB, Chroma, or Milvus Lite. None of them need a separate service to run, and all three offer an HNSW-family index. AnythingLLM’s default vector store on a fresh install is LanceDB for exactly this reason, as the RAG setups guide notes.

A household or a small team, hundreds of thousands to low millions of chunks. Qdrant single-node, Milvus Standalone, or Weaviate single-node on Docker. Each is one container with a persistent volume; each has an HNSW index with tunable ef and ef_construction knobs. None of them need a cluster at this scale, and pgvector on a beefy Postgres box also fits if the rest of your data already lives there. RAM is the ceiling, not disk: a million 768-dim float32 vectors are 3.07 GB (2.86 GiB) of raw data, and the HNSW graph comes on top.

Multi-user production, tens of millions of chunks or strict latency SLAs. Milvus Distributed or Weaviate on Kubernetes. Milvus adds the on-disk DiskANN index, SCANN, and the CAGRA GPU index; Weaviate adds the disk-based HFresh alongside HNSW, Flat, and Dynamic. pgvector is the wrong tool here unless the dataset still fits on one machine; horizontal sharding on Postgres is a separate engineering project.

The catches

Changing embedders means re-embedding everything. This is true for every database in the table, not just one of them. Vectors from one embedding model are not in the same space as vectors from another, and the database cannot tell you that a stored vector was produced by a now-deprecated model unless you tagged it yourself. AnythingLLM, Open WebUI, RAGFlow and Cherry Studio all handle this in their own way; what they cannot do is avoid the work. Plan for a re-embed step whenever you swap models.

Weaviate’s licence split is easy to miss. The repository’s LICENSE file states that code under wl/ is “only available under a separate enterprise license”, while the rest of the codebase is BSD-3-Clause, and that “The BSD license does not grant the right to use these features or circumvent any license key.” The same file dates the wl/ copyright to Weaviate BV, 2026-present.

pgvector still eats RAM for big indexes. The extension is in Postgres, which means transactional safety, JOINs, and backups come for free, but the README notes that HNSW indexes build significantly faster when the graph fits into maintenance_work_mem. A million 768-dim float32 vectors are 3.07 GB (2.86 GiB) of raw data before the index graph (default m=16, ef_construction=64) and the heap. If your embedder produces 1024-dim or 1536-dim vectors, multiply accordingly.

Chroma’s “latest” tag ships a dev version. The releases page marks the latest tag as a pre-release that currently ships 1.5.10.dev326, while the newest numbered release is 1.5.9 (5 May 2026); pin to a specific version in production so the next dev build does not walk into your deployment.

What to actually build

Laptop, one user, personal document pile: LanceDB (embedded) or Chroma (persistent disk). The choice is library vs daemon. AnythingLLM’s default is LanceDB, and AnythingLLM’s native reranker only works with LanceDB; Chroma is the Open WebUI default. Both are Apache-2.0 and both run on CPU.

Postgres shop that wants SQL and vectors in one place: pgvector with an HNSW index, vector type matching your embedder, cosine distance unless you have a specific reason. Pin the ef_construction and m values in the migration so the index is reproducible across environments.

Multi-user, multi-language, plan to grow: Qdrant single-node on a small host; move to Milvus Distributed or Weaviate on Kubernetes when the dataset passes a few million chunks or the latency budget tightens.

Don’t pick a database before picking an embedder. The embedder sets the vector dimensionality, the distance metric, and (often) the reranker model; the database has to match. The by-VRAM guide linked at the top is the one to read alongside this page; the RAG setups guide above is the one to read after both choices are made.