Vector database

A vector database stores texts, images or products as lists of numbers that capture their meaning, called embeddings, and finds the ones closest to a question. A webshop search can then find "warm boots for winter" when the product text says "insulated snow shoes", and a chatbot can find the right passage in your manuals. In a RAG system, it finds the passages the AI answers from.

How does it work?

A vector database stores each embedding with its content and metadata, such as the source document, a date or a product category. A question is turned into a vector with the same embedding model, and the database returns the stored vectors closest to it, measured with a distance such as cosine similarity. A traditional database mainly looks up records by exact match; a vector database searches by similarity (Wikipedia).

The exact answer means comparing the question with every stored vector, which the authors of Meta's Faiss library call brute-force search: the work grows with every vector you add. Vector databases therefore use approximate nearest neighbour (ANN) indexes, which examine only part of the collection and can miss some true neighbours:

• HNSW (Malkov and Yashunin, 2016) links the vectors in a graph with several layers. A search starts in the sparse top layer and works down, with effort that grows logarithmically with the number of vectors.
• IVF (inverted file) groups the vectors into clusters when they are indexed and searches only the clusters nearest to the question (Douze et al., 2024).
• Product quantization (PQ) splits each vector into parts and stores a short code per part, so vectors take less memory at some cost in accuracy, a form of quantization.

The Faiss paper describes the choice as a trade-off between speed, memory and accuracy. Graph indexes suit collections of up to about a million vectors when memory is no constraint. Beyond ten million vectors, the time to build them becomes the limiting factor, and for collections that only fit in memory when compressed, IVF indexes are the only option.

Which kinds are there?

There are dedicated vector databases such as Pinecone, Weaviate, Qdrant, Milvus and Chroma, libraries such as Faiss, and extensions to existing databases. pgvector, for example, adds vector search to PostgreSQL, so the vectors sit "with the rest of your data" and can be filtered with ordinary SQL.

Where does it go wrong?

• Results change with the index. By default pgvector searches exactly, with perfect recall. Its documentation warns that after adding an approximate index "you will see different results for queries".
• Filters and indexes clash. In pgvector, a filter is applied after the approximate index has been scanned. If a condition matches 10% of the rows, an HNSW search with the default setting of 40 candidates returns only about 4 matching rows on average, unless iterative index scans are switched on (documentation, September 2026).
• Exact terms. Vector search can miss exact terms such as an error code (semantic search). pgvector suggests combining it with PostgreSQL full-text search and merging the results, for example with reciprocal rank fusion.
• One model only. Stored vectors and questions must come from the same embedding model, so switching models means embedding everything again.

Example

Suppose a webshop with 3,000 products wants a search that understands "something warm for a winter walk". It embeds each product description once and stores the vectors in PostgreSQL with pgvector, next to price and stock. Each search is embedded with the same model and returns the ten nearest products that are in stock. With a few thousand products the shop can start with pgvector's default exact search. An HNSW index becomes useful when the catalogue grows, and from then on the results may differ slightly from the exact ones.