The best vector database for self-hosted RAG
For most self-hosted RAG systems, Qdrant is the best default: fast, filters well at query time, and straightforward to operate. Use pgvector if your data already lives in Postgres and you would rather not run a second database, Weaviate when you need hybrid keyword-and-vector search, and Milvus when collections reach hundreds of millions of vectors.
At a glance
| Strength | Watch out for | Hosted here from | |
|---|---|---|---|
| Qdrant | Filtering at query time, quantization, easy ops | Fewer built-in ingestion helpers | $5.99/mo |
| Weaviate | Hybrid search, vectorizer modules | Heavier stack | $11.99/mo |
| Milvus | Very large collections, index variety | Most moving parts to operate | $18.99/mo |
| Chroma | Simplest API, near-zero configuration | Not built for huge corpora | $5.99/mo |
| pgvector | No second database, SQL joins, one backup | Slower past a few million vectors | $5.99/mo |
Start by asking whether you need one at all
The most common mistake in a first RAG build is standing up a dedicated vector database on day one. If your documents already live beside relational data, pgvector keeps everything in Postgres: embeddings sit next to the rows they describe, joins still work, permissions and backups do not change, and you avoid operating a second system. Up to a few million vectors this is usually the right call.
Move to a dedicated store when one of three things happens: the index no longer fits comfortably in memory, several applications need to share it, or you want to query it directly with filters that SQL makes awkward.
Filtering is the feature that decides most projects
Retrieval in production is rarely “find similar documents”. It is “find similar documents belonging to this customer, of this type, since March”. A store that applies filters during search returns the right results; one that filters afterwards returns fewer results than you asked for, or none.
This is where Qdrant earns its default position: payload filtering is first-class and fast. Weaviate handles it well too and adds hybrid search, which matters when exact keyword matches must not be diluted by semantic similarity — product codes, error strings, names.
Sizing, honestly
A rough figure worth keeping: a million 768-dimension vectors occupy roughly 3 GB before quantization, and noticeably less with it. Scalar quantization typically cuts memory around fourfold at a small recall cost.
So a corpus of 200,000 chunks — a large internal documentation set — is comfortably a small plan, not an infrastructure project. Most teams over-provision this by an order of magnitude.
What about Pinecone?
Pinecone is a hosted service, and a good one: if nobody on your team wants to think about this layer at all, paying for it is defensible. What you give up is where the embeddings live, the ability to query the store directly, and a cost that moves with usage rather than staying flat.
Every option in the table above runs on a single-tenant instance here, which is the self-hosted answer to the same question.
The verdict
Default to Qdrant. Choose pgvector if you already run Postgres and the corpus is modest — it is the option with the least new operational surface. Choose Weaviate for hybrid search, Milvus at very large scale, and Chroma while you are still prototyping and do not want to think about any of this yet.
Mentioned on this page
Qdrant
A fast vector database with real filtering, written in Rust.
from $5.99/moDetails →
Weaviate
A vector database with hybrid search and built-in vectorization modules.
from $11.99/moDetails →
Milvus
Vector search built for very large collections.
from $18.99/moDetails →
Chroma
A lightweight embedding store that gets out of the way.
from $5.99/moDetails →Questions
Can I switch vector databases later?
Do I need a GPU for embeddings?
Which one do your own customers pick most?
Put your AI stack on your own box
Pick an app, pick a size, and have it running today. Month to month, cancel whenever.