Six self-hosted vector databases will store the embeddings for a local RAG stack this afternoon. The cluster already covers the orchestration layer in the RAG setups guide and the embedder and reranker model choices in the by-VRAM embedding guide. This page covers what sits underneath both: the database that holds the vectors and runs the nearest-neighbour search at query time. Picking it is the first decision that locks in how your stack scales, how it fails, and what you have to rebuild when you change embedders.
The comparison
| Database | Licence | Language | Latest verified release | Deployment modes | Indexing | SDKs |
|---|---|---|---|---|---|---|
| Qdrant | Apache 2.0 (Qdrant Solutions GmbH, 2026) | Rust | v1.20.0 (2026-10-22) | Client-server; Qdrant Edge embedded variant | HNSW, payload indexes, ACORN filter | Python, JS/TS, Rust, Go, .NET, Java |
| Milvus | Apache 2.0 | Go and C++ | v3.0.5 (2026-10-30) | Lite (embedded Python), Standalone (Docker), Distributed (Kubernetes) | HNSW, IVF, FLAT, SCANN, DiskANN, CAGRA GPU index | Python (PyMilvus), Go, Java, Node.js, C++, C# (Microsoft-contributed) |
| Chroma | Apache 2.0 | Rust | 1.5.9 (2026-05-05); the latest pre-release tag ships 1.5.10.dev326 | In-memory, persistent disk, client-server; Chroma Cloud (hosted, serverless) | HNSW | Python, JS/TS |
| Weaviate | BSD-3-Clause for code outside wl/; a separate Weaviate enterprise licence for code inside wl/ (Weaviate BV, 2026-present) | Go | v1.40.3 (2026-10-28) | Embedded (experimental), Docker, Kubernetes, Weaviate Cloud | HNSW, Flat, Dynamic, HFresh (HFresh only supports cosine and l2-squared) | Python, JS/TS, Java, Go, C#/.NET |
| pgvector | PostgreSQL License (BSD-style) | C | v0.8.7 | A PostgreSQL extension; no separate server | HNSW, IVFFlat; vector (2,000 dim indexed), halfvec (4,000), bit (64,000), sparsevec (1,000 non-zero) | Any language with a Postgres client; the README lists libraries or examples for 35 language rows |
| LanceDB | Apache 2.0 | Rust core, multi-language wrappers | v0.41.0 (2026-10-29) | Embedded (library, runs in-process); LanceDB Enterprise (distributed) | IVF-PQ, IVF-HNSW; GPU index building | Python, JS/TS, Java, Rust |
All six are Apache-2.0 or BSD-family permissive except Weaviate, whose
weaviate/ repository is split: the BSD-3-Clause licence applies to the OSS
tree, and code under the wl/ directory is “only available under a separate
enterprise license”, with enterprise features enabled by licence key. Readers who care about redistribution should read the
Weaviate LICENSE file
directly before packaging the binary.
Picking by what you already have
The cheapest pick is the database that piggy-backs on infrastructure you are already running. Each entry below is a one-line answer; the next section covers when those answers are wrong.
You already run PostgreSQL. pgvector. v0.8.7 ships with HNSW and IVFFlat
indexes, four vector types (vector, halfvec, bit, sparsevec), six
distance metrics (L2, inner product, cosine, L1, Hamming, Jaccard), and full
Postgres features (ACID, point-in-time recovery, JOINs), plus hybrid search with
Postgres full-text search and binary quantization. Everything is in the same database as your application
data, so embedding metadata, user permissions, and backups already work.
Cost: vector indexes still eat RAM; a million 768-dim float32 vectors are
3.07 GB (2.86 GiB) of raw data before the HNSW graph and the table heap.
You write Python and want zero moving parts. LanceDB (embedded) or Milvus
Lite. Both run inside your Python process: import lancedb and the database
file lives next to your script; pymilvus in Lite mode is the same idea with a
different on-disk format. LanceDB documents IVF-PQ and IVF-HNSW indexes plus
GPU index building; the current Milvus Lite documentation lists FLAT,
BRUTE_FORCE, IVF_FLAT, IVF_SQ8, HNSW, HNSW_SQ and AUTOINDEX.
You want a single server you can drop on a host. Qdrant. One
Docker image started with
docker run -p 6333:6333 qdrant/qdrant; the HNSW index, payload indexes, and
the ACORN search algorithm (which Qdrant describes as improving accuracy with
multiple high-cardinality filters) are all in the box. It is written in Rust,
and the official SDK list (Python, JS/TS, Rust, Go, .NET, Java) covers the
usual stacks.
You want Kubernetes and billion-scale from day one. Milvus Distributed. v3.0.5 (released 2026-10-30) is the current line; the docs list Lite, Standalone (single Docker image), and Distributed (Kubernetes, cloud-native). Indexes cover the full menu: HNSW, IVF, FLAT, SCANN, DiskANN, and the NVIDIA CAGRA GPU index. Weaviate on Kubernetes, below, is the other option at this scale.
You want vectorizers baked in. Weaviate. Its model-provider integrations
(Cohere, OpenAI, Hugging Face, Voyage AI, and others) can generate embeddings
at insert and search time, so the application code only sends text. v1.40.3
(released 2026-10-28) ships HNSW, Flat, Dynamic and HFresh; HFresh is a
disk-based, cluster-based index, and it only supports cosine and l2-squared
distance. The trade is the licence split mentioned above: code outside wl/
is BSD-3, code inside it is under a separate enterprise licence.
You want a managed option with a self-hosted fallback. Weaviate Cloud or Chroma Cloud. Both have self-hosted paths and hosted paths. Chroma’s documentation covers in-memory, persistent and client-server modes for self-hosted runs, and single-node Chroma collections use an HNSW index.
Picking by scale
The same database behaves very differently across the three bands a local stack actually hits.
One user, one laptop, tens of thousands of chunks. LanceDB, Chroma, or Milvus Lite. None of them need a separate service to run, and all three offer an HNSW-family index. AnythingLLM’s default vector store on a fresh install is LanceDB for exactly this reason, as the RAG setups guide notes.
A household or a small team, hundreds of thousands to low millions of
chunks. Qdrant single-node, Milvus Standalone, or Weaviate single-node on
Docker. Each is one container with a persistent volume; each has an HNSW
index with tunable ef and ef_construction knobs. None of them need a
cluster at this scale, and pgvector on a beefy Postgres box also fits if the
rest of your data already lives there. RAM is the ceiling, not disk: a million
768-dim float32 vectors are 3.07 GB (2.86 GiB) of raw data, and the HNSW graph
comes on top.
Multi-user production, tens of millions of chunks or strict latency SLAs. Milvus Distributed or Weaviate on Kubernetes. Milvus adds the on-disk DiskANN index, SCANN, and the CAGRA GPU index; Weaviate adds the disk-based HFresh alongside HNSW, Flat, and Dynamic. pgvector is the wrong tool here unless the dataset still fits on one machine; horizontal sharding on Postgres is a separate engineering project.
The catches
Changing embedders means re-embedding everything. This is true for every database in the table, not just one of them. Vectors from one embedding model are not in the same space as vectors from another, and the database cannot tell you that a stored vector was produced by a now-deprecated model unless you tagged it yourself. AnythingLLM, Open WebUI, RAGFlow and Cherry Studio all handle this in their own way; what they cannot do is avoid the work. Plan for a re-embed step whenever you swap models.
Weaviate’s licence split is easy to miss. The repository’s LICENSE file
states that code under wl/ is “only available under a separate enterprise
license”, while the rest of the codebase is BSD-3-Clause, and that “The BSD
license does not grant the right to use these features or circumvent any
license key.” The same file dates the wl/ copyright to Weaviate BV,
2026-present.
pgvector still eats RAM for big indexes. The extension is in Postgres,
which means transactional safety, JOINs, and backups come for free, but the
README notes that HNSW indexes build significantly faster when the graph fits
into maintenance_work_mem. A million 768-dim float32 vectors are 3.07 GB
(2.86 GiB) of raw data before the index graph (default m=16, ef_construction=64) and the heap. If your embedder produces 1024-dim or
1536-dim vectors, multiply accordingly.
Chroma’s “latest” tag ships a dev version. The releases page marks the
latest tag as a pre-release that currently ships 1.5.10.dev326, while the
newest numbered release is 1.5.9 (5 May 2026); pin to a specific version in
production so the next dev build does not walk into your deployment.
What to actually build
Laptop, one user, personal document pile: LanceDB (embedded) or Chroma (persistent disk). The choice is library vs daemon. AnythingLLM’s default is LanceDB, and AnythingLLM’s native reranker only works with LanceDB; Chroma is the Open WebUI default. Both are Apache-2.0 and both run on CPU.
Postgres shop that wants SQL and vectors in one place: pgvector with an
HNSW index, vector type matching your embedder, cosine distance unless you
have a specific reason. Pin the ef_construction and m values in the
migration so the index is reproducible across environments.
Multi-user, multi-language, plan to grow: Qdrant single-node on a small host; move to Milvus Distributed or Weaviate on Kubernetes when the dataset passes a few million chunks or the latency budget tightens.
Don’t pick a database before picking an embedder. The embedder sets the vector dimensionality, the distance metric, and (often) the reranker model; the database has to match. The by-VRAM guide linked at the top is the one to read alongside this page; the RAG setups guide above is the one to read after both choices are made.