Six self-hosted vector databases will store the embeddings for a local RAG stack this afternoon. The cluster already covers the orchestration layer in the RAG setups guide and the embedder and reranker model choices in the by-VRAM embedding guide. This page covers what sits underneath both: the database that holds the vectors and runs the nearest-neighbour search at query time. Picking it is the first decision that locks in how your stack scales, how it fails, and what you have to rebuild when you change embedders.
The comparison
| Database | Licence | Language | Latest verified release | Deployment modes | Indexing | SDKs |
|---|---|---|---|---|---|---|
| Qdrant | Apache 2.0 (Qdrant Solutions GmbH, 2026) | Rust | v1.19.2 (2026-10-05) | Client-server; Qdrant Edge embedded variant | HNSW, payload indexes, ACORN filter | Python, JS/TS, Rust, Go, .NET, Java |
| Milvus | Apache 2.0 | Go (with C++/Python in stack) | v3.0.3 (2026-10-14) | Lite (embedded Python), Standalone (Docker), Distributed (Kubernetes) | HNSW, IVF, FLAT, SCANN, DiskANN, CAGRA GPU index | Python (PyMilvus), Go, Java, Node.js, C++, C# (Microsoft-contributed) |
| Chroma | Apache 2.0 | Rust | 1.5.x line (the latest tag ships 1.5.10.dev324) | In-memory, persistent disk, client-server; Chroma Cloud (managed, serverless) | HNSW | Python, JS/TS |
| Weaviate | BSD-3-Clause for code outside wl/; Weaviate Enterprise License for code inside wl/ (Weaviate BV, 2026) | Go | v1.40.1 (2026-10-13) | Embedded (Python or JS/TS), Docker, Kubernetes, Weaviate Cloud | HNSW, Flat, Dynamic, HFresh (HFresh only supports cosine and l2-squared) | Python, JS/TS, Go, Java |
| pgvector | PostgreSQL License (BSD-style) | C | v0.8.7 | A PostgreSQL extension; no separate server | HNSW, IVFFlat; vector (2,000 dim indexed), halfvec (4,000), bit (64,000), sparsevec (1,000 non-zero) | Whatever speaks Postgres: 40+ client libraries |
| LanceDB | Apache 2.0 | Rust core, multi-language wrappers | v0.40.0 (2026-10-07) | Embedded (library, runs in-process); LanceDB Cloud (serverless) | IVF-PQ, HNSW; GPU index building | Python, JS/TS, Java, Rust |
All six are Apache-2.0 or BSD-family permissive except Weaviate, whose
weaviate/ repository is split: the BSD-3-Clause licence applies to the OSS
tree, and code under the wl/ directory falls under a separate Weaviate
Enterprise License. Readers who care about redistribution should read the
Weaviate LICENSE file
directly before packaging the binary.
Picking by what you already have
The cheapest pick is the database that piggy-backs on infrastructure you are already running. Each entry below is a one-line answer; the next section covers when those answers are wrong.
You already run PostgreSQL. pgvector. v0.8.7 ships with HNSW and IVFFlat
indexes, four vector types (vector, halfvec, bit, sparsevec), six
distance metrics (L2, inner product, cosine, L1, Hamming, Jaccard), and full
Postgres features (ACID, point-in-time recovery, JOINs, full-text hybrid search,
binary quantization). Everything is in the same database as your application
data, so embedding metadata, user permissions, and backups already work.
Cost: vector indexes still eat RAM; a million 768-dim float32 vectors at HNSW
is well into the tens of GB.
You write Python and want zero moving parts. LanceDB (embedded) or Milvus
Lite. Both run inside your Python process: import lancedb and the database
file lives next to your script; pymilvus in Lite mode is the same idea with a
different on-disk format. LanceDB’s v0.40.0 release ships IVF-PQ and HNSW
indexes plus GPU index building; Milvus Lite inherits the same HNSW/IVF/FLAT
index family as the full server.
You want a single Go binary you can drop on a host. Qdrant. One Docker image, one port, one config file; the HNSW index, payload indexes, and the ACORN delta-filter mechanism for “multiple high-cardinality filters” are all in the box. The Rust implementation is fast on commodity hardware and the SDK list (Python, JS/TS, Rust, Go, .NET, Java) covers the usual stacks.
You want Kubernetes and billion-scale from day one. Milvus Distributed.
v3.0.3 (released 2026-10-14) is the current line; the docs list Lite,
Standalone (single Docker image), and Distributed (Kubernetes, cloud-native).
Indexes cover the full menu: HNSW, IVF, FLAT, SCANN, DiskANN, and the NVIDIA
CAGRA GPU index. The weaviate/ option below is roughly comparable at this
scale; Milvus tends to be picked when the team already standardises on K8s.
You want vectorizers baked in. Weaviate. Its module system can pull an
embedding model from a remote vectorizer service (Cohere, OpenAI, Hugging
Face, Voyage, and others) at insert and search time, so the application code
only sends text. v1.40.1 (released 2026-10-13) ships HNSW, Flat, Dynamic and
HFresh; HFresh is the newest, and it only supports cosine and l2-squared
distance. The trade is the licence split mentioned above: code outside wl/
is BSD-3, code inside it is the proprietary Weaviate Enterprise License.
You want a managed option that still keeps the data in your VPC. Weaviate Cloud (WCD) or Chroma Cloud. Both have self-hosted paths and managed paths; Chroma Cloud is the newer of the two. The Chroma repository documents in-memory, persistent disk, and client-server modes for self-hosted runs, with the HNSW index default.
Picking by scale
The same database behaves very differently across the three bands a local stack actually hits.
One user, one laptop, tens of thousands of chunks. LanceDB, Chroma, or Milvus Lite. None of them need a separate service to run, none of them need RAM beyond what your chat model already has, and all three ship a usable HNSW default. AnythingLLM’s default vector store on a fresh install is LanceDB for exactly this reason, as the RAG setups guide notes.
A household or a small team, hundreds of thousands to low millions of
chunks. Qdrant single-node, Milvus Standalone, or Weaviate single-node on
Docker. Each is one container with a persistent volume; each has an HNSW
default with sane ef and ef_construction knobs. None of them need a
cluster at this scale, and pgvector on a beefy Postgres box also fits if the
rest of your data already lives there. RAM is the ceiling, not disk: a million
768-dim float32 vectors in HNSW is the high tens of GB.
Multi-user production, tens of millions of chunks or strict latency SLAs. Milvus Distributed or Weaviate on Kubernetes. Milvus adds DiskANN and SCANN for cold storage on SSD and the CAGRA GPU index for hot retrieval; Weaviate adds HFresh, HNSW, Flat, and Dynamic. pgvector is the wrong tool here unless the dataset still fits on one machine; horizontal sharding on Postgres is a separate engineering project.
The catches
Changing embedders means re-embedding everything. This is true for every database in the table, not just one of them. Vectors from a 768-dim MiniLM model are not in the same space as vectors from a 1024-dim BGE model, and the database cannot tell you that a stored vector was produced by a now-deprecated model unless you tagged it yourself. AnythingLLM, Open WebUI, RAGFlow and Cherry Studio all handle this in their own way; what they cannot do is avoid the work. Plan for a re-embed step whenever you swap models.
Weaviate’s licence split is easy to miss. The repository’s LICENSE file
states that code under wl/ is under the Weaviate Enterprise License, while
the rest of the codebase is BSD-3-Clause. If you build and redistribute
Weaviate, you are shipping a binary that mixes the two, and the wl/ portion
is the one to read. The same file dates the copyright to Weaviate BV, 2026.
pgvector still eats RAM for big indexes. The extension is in Postgres,
which means transactional safety, JOINs, and backups come for free, but the
HNSW index lives in shared_buffers. A million 768-dim float32 vectors at
default m=16, ef_construction=64 is comfortably into the tens of GB before
you count the heap. If your embedder produces 1024-dim or 1536-dim vectors,
multiply accordingly.
Chroma’s “latest” tag ships a dev version. The releases page lists
1.5.10.dev324 as the latest tag; pin to a specific version in production
so the next dev build does not walk into your deployment. The repository is
clearly maintained and the 1.5.x line is current, but pinning is the right
move for anything that needs to reproduce.
LanceDB Cloud is the only fully-managed offering in this list that is not vendor-locked to Weaviate or Chroma. LanceDB Cloud is serverless and the on-disk file format is the same one the embedded library writes, so moving back to self-hosted is a file copy. The reverse is not always true for vendors whose managed service sits on top of a forked engine.
What to actually build
Laptop, one user, personal document pile: LanceDB (embedded) or Chroma (persistent disk). The choice is library vs daemon. AnythingLLM’s default is LanceDB and it ships a reranker in that pairing; Chroma is the Open WebUI default and is the broader API option. Both are MIT/Apache-2.0, both run on CPU, both have HNSW.
Postgres shop that wants SQL and vectors in one place: pgvector with an
HNSW index, vector type matching your embedder, cosine distance unless you
have a specific reason. Pin the ef_construction and m values in the
migration so the index is reproducible across environments.
Multi-user, multi-language, plan to grow: Qdrant single-node on a small host; move to Milvus Distributed or Weaviate on Kubernetes when the dataset passes a few million chunks or the latency budget tightens. The Rust and Go implementations do not need a GPU.
Don’t pick a database before picking an embedder. The embedder sets the vector dimensionality, the distance metric, and (often) the reranker model; the database has to match. The by-VRAM guide linked at the top is the one to read alongside this page; the RAG setups guide above is the one to read after both choices are made.