What is a vector database?

A vector database stores objects as fixed-length arrays of floating-point numbers — typically 384, 768, 1024, 1536, or 3072 dimensions — and indexes them so that the nearest neighbors of a query vector can be returned in milliseconds. The “vector” comes from an embedding model: a neural network that maps text, images, audio, or code into a point in a high-dimensional space where semantically similar inputs land near one another. OpenAI’stext-embedding-3-largeoutputs 3072-dimensional vectors; Cohere’sembed-v3outputs 1024; Google’stext-embedding-004outputs 768; the open-sourceall-MiniLM-L6-v2(Reimers & Gurevych, “Sentence-BERT,” arXiv:1908.10084) outputs 384.

The defining operation is approximate nearest neighbor (ANN) search. Exact nearest neighbor over a billion 768-dimensional vectors is computationally infeasible at query time, so vector databases use indexes that trade a small amount of recall for orders-of-magnitude faster lookup. The two dominant families are HNSW (Hierarchical Navigable Small World graphs, Malkov & Yashunin, arXiv:1603.09320, 2016) and IVF (inverted file with product quantization, Jégou et al., “Product Quantization for Nearest Neighbor Search,” IEEE TPAMI 2011). Facebook AI Research’s FAISS library (Johnson, Douze & Jégou, arXiv:1702.08734) is the reference implementation that most vector databases either embed directly or borrow algorithms from.

The current generation of vector databases emerged around 2022–2023 with the rise of large language models. Pinecone (founded 2019) launched a fully managed serverless tier in 2024. Weaviate (open-source, Apache 2.0) added native hybrid search combining BM25 with vector similarity. Milvus (a CNCF graduated project as of June 2024) supports billions of vectors with GPU-accelerated indexing. Qdrant (written in Rust, Apache 2.0) emphasizes payload filtering. Chroma is the lightweight in-process choice favored by LangChain and LlamaIndex prototypes. pgvector (PostgreSQL extension, MIT license) reached version 0.7.0 in April 2024 with HNSW support and is the path for teams that want vector search inside their existing Postgres without operating a second system.

The dominant production use case is retrieval-augmented generation (RAG), introduced by Lewis et al. (“Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” arXiv:2005.11401, NeurIPS 2020). In RAG, a user query is embedded, the vector database returns the top-k most similar document chunks, and those chunks are passed as context to an LLM. This grounds the LLM’s response in retrieved evidence and reduces hallucination on domain-specific or recent knowledge that the model was not trained on.

Distance metrics matter and are not interchangeable. Cosine similarity is the standard for normalized text embeddings; Euclidean (L2) distance is common for image embeddings; dot product is used when embeddings are not normalized and magnitude carries signal. Using the wrong metric for the embedding model you’re querying produces ranked results that look plausible but are subtly wrong — a class of bug that survived into production at multiple vendors before being caught.

Vector databases are not a replacement for relational or document databases. They are a complement: the embedding is a pointer-by-meaning, while the source-of-truth row still lives in Postgres, S3, or a content store. Most production architectures store the canonical document in a primary store and the embedding plus a foreign-key payload in the vector index.