Vector DB

How it works

First, an AI model turns documents or images into lists of numbers called embeddings. The more similar in meaning two pieces of data are, the closer their number lists end up. A vector DB stores these vectors, converts a query into a vector the same way, then finds and returns the closest vectors. When there is a great deal of data, comparing every item one by one is impractical, so 'approximate nearest neighbor search', which gives up a little accuracy for a large gain in speed, is mainly used. A vector DB adds features of an ordinary database, such as insert, update and delete and condition (metadata) filters, on top of this.

How it has evolved

The core of a vector DB is 'how to find nearby vectors quickly', and research on this has steadily built up. The 2016 HNSW paper by Malkov and Yashunin proposed a new search method that links data into a multi-layer graph, taking big steps in the upper layers and searching in fine detail in the lower layers. The 2017 paper by Johnson, Douze and Jégou used GPUs efficiently to make similarity search possible even across a billion vectors, and released this work publicly. This research led to Faiss, an open-source library developed mainly at Meta AI Research. Since then, dedicated vector DBs that bundle these search techniques with storage and management features have been used alongside extensions that add vector search to existing databases (e.g., Postgres).

Why it matters

LLMs do not know internal documents or recent information outside what they were trained on, so RAG, which finds the needed material and passes it to the model along with the question, is widely used. The vector DB is what finds the document chunks related to the question. Because it finds content that is similar in meaning even when the words do not match exactly, asking about 'how to get a refund' can turn up a 'return procedure' document. It is also used as 'long-term memory' storage that lets AI agents remember past conversations and task history.

Things to know

Because the search is approximate, it can occasionally miss the closest result, and settings have to be tuned to balance speed and accuracy. Search quality depends far more on which embedding model is used and how documents are split (chunking) than on the vector DB itself. Meaning-based search can be weak when exact word matches matter, such as product codes or proper nouns, so it is often combined with hybrid search, which mixes in keyword search, or with a reranker that reorders the results. If the data is small, an extension to an existing database may be enough, without a dedicated vector DB.

Example

Suppose a company builds a chatbot that answers questions based on its internal policy documents. If the documents are split into paragraphs, embedded and stored in a vector DB, then when an employee asks a question, the few most relevant paragraphs can be found and passed to the LLM along with it. The company can use a dedicated service such as Pinecone, or, if it already uses Postgres, install the open-source extension pgvector to store and search vectors alongside its existing data. Teams building their own search engine sometimes use a library such as Faiss.

Related terms

Embedding RAG Semantic search Chunking Hybrid search Reranker

References

• Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs (2016)
• Billion-scale similarity search with GPUs (2017)
• What is a Vector Database & How Does it Work? Use Cases + Examples | Pinecone
• Vector database - Wikipedia
• GitHub - pgvector/pgvector: Open-source vector similarity search for Postgres · GitHub
• Home · facebookresearch/faiss Wiki · GitHub