Skip to content

Vector Database

Also called: Vector store · Vector index · Vector search engine

Definition
A database that stores embeddings and quickly finds the most similar ones, using approximate nearest-neighbour indexes such as HNSW or IVF.

Exhibit A

Vector search or keyword search?

8 / 8

Vector search

Finds what the customer means

Keyword search

Finds what the customer typed

  1. Vector searchMeaning — nearby vectors
    Keyword searchShared words, weighted by frequency (BM25)
  2. Vector searchFound: same meaning, no shared words
    Keyword searchMissed: no words in common
  3. Vector searchWeak: similar numbers look alike
    Keyword searchExact match
  4. Vector searchANN graph or clusters (HNSW, IVF)
    Keyword searchInverted index of terms
  5. Vector searchCosine similarity or distance
    Keyword searchTerm-frequency relevance score
  6. Vector searchNear-synonyms and look-alike codes
    Keyword searchParaphrases and other languages
  7. Vector searchQuestions, paraphrases, multilingual text
    Keyword searchCodes, SKUs, names, exact phrases

Verdict

Use both: run vector and keyword search together, fuse the rankings (hybrid search), and filter by tenant and language before anything reaches the model.

Compare how vector and keyword search handle the same support questions, from mixed-language queries to exact order IDs — and why production bots usually run both.

Verdict: Use both: run vector and keyword search together, fuse the rankings (hybrid search), and filter by tenant and language before anything reaches the model.

What is a vector database?

A vector database is a data store built to index vector embeddings — lists of hundreds or thousands of numbers that represent the meaning of a piece of text, an image or audio — and to answer one question fast: which stored vectors are closest to this query vector? In a WhatsApp bot it is the retrieval layer behind retrieval-augmented generation (RAG): help-article chunks are stored as vectors, the customer's question is embedded, and the database returns the passages closest in meaning, even when they share no words with the question.

Comparing a query with every stored vector (exact or "flat" search) is accurate but slows down linearly with size. Vector databases instead build approximate nearest-neighbour (ANN) indexes. HNSW (Hierarchical Navigable Small World) links vectors into a layered graph that a search walks greedily towards the query; IVF (inverted file) clusters vectors and searches only the clusters nearest the query; product quantisation compresses vectors to save memory. Each trades a little recall — occasionally missing a true neighbour — for much faster queries, tuned with parameters such as HNSW's ef_search or IVF's nprobe.

The database is infrastructure, not content. The knowledge base is what you index — curated policies, FAQs and product facts — and the vector database is where its embeddings live and get searched. It also differs from keyword search engines, which rank documents by matching terms (BM25): vectors find an article titled "Delivery timelines" for "parcel kab aayega", while keywords find order 40718829 exactly. Production systems increasingly use both, in a dedicated vector database or a vector index inside a database they already run.

Key takeaways

  • A vector database stores embeddings and returns the nearest ones to a query vector in milliseconds.
  • It uses approximate nearest-neighbour (ANN) indexes such as HNSW or IVF, trading a little recall for large speed gains.
  • Metadata filters — tenant, language, product, validity — matter as much as similarity for correct answers.
  • Keyword search still wins on exact codes such as SKUs and order IDs; hybrid search combines both.
  • Raw storage is dimensions × 4 bytes per vector: a million 1,536-dimension vectors is about 6 GB before index overhead.

How does a vector database work?

  1. Fix the embedding model and metric

    Use one embedding model per index; its dimension (for example 768 or 1,536) defines the vector column. Use the distance metric the model was trained for — usually cosine, or dot product on normalised vectors.

  2. Store vectors with metadata

    Each record holds the vector, the source text or a pointer to it, and filterable fields: tenant or WhatsApp Business Account, language, product, updated_at, access level.

  3. Build an ANN index

    HNSW gives low-latency, high-recall search at the cost of memory and slower builds; IVF builds faster and uses less memory but needs more tuning. Small collections can simply stay on exact search.

  4. Query with filters

    Embed the query and search for the top k with metadata filters applied during the search, not after it. Optionally run keyword search in parallel and fuse the two rankings.

  5. Measure recall and latency

    Compare ANN results with exact search on a sample to measure recall@k, and watch P99 latency under realistic load. Raise ef_search or nprobe for recall, lower them for speed.

A vector database in code

-- One row per knowledge-base chunk, with filterable metadata.
CREATE TABLE kb_vectors (
  id         bigserial    PRIMARY KEY,
  tenant_id  text         NOT NULL,      -- one business per tenant
  lang       text         NOT NULL,      -- 'en', 'hi', 'mr' …
  body       text         NOT NULL,
  embedding  vector(1536) NOT NULL       -- dimension fixed by the embedding model
);
 
-- HNSW index for cosine distance (m and ef_construction at their defaults).
CREATE INDEX kb_vectors_hnsw ON kb_vectors
  USING hnsw (embedding vector_cosine_ops) WITH (m = 16, ef_construction = 64);
 
-- Query time: a higher ef_search means better recall and slower queries.
SET hnsw.ef_search = 100;
 
SELECT id, body, embedding <=> $1 AS distance
FROM kb_vectors
WHERE tenant_id = $2 AND lang = $3
ORDER BY embedding <=> $1
LIMIT 5;

A vector index inside an existing database. With very selective filters, check how your engine combines filtering with the ANN scan.

Why does a vector database matter?

Retrieval quality sets the ceiling for an AI bot's answers: if the right passage is not in the top few results, the model cannot use it. The vector database decides how fast that retrieval is, how it scales from a thousand chunks to millions, whether results can be restricted to one business, language or product line, and where the indexed data physically sits — a data residency question for teams handling customer information under the DPDP Act. Choosing and tuning it is an engineering decision with direct effects on answer accuracy, latency and cost.

Common mistakes with a vector database

  • Filtering after the search

    If metadata filters run after the ANN step, a selective filter (one tenant, one language) can leave fewer than k results, or none. Use filter-aware search, partitions per tenant, or a larger candidate pool.

  • Mixing embedding models

    Vectors from different models — or different versions of one model — are not comparable. Store the model name with each vector and re-embed the whole collection when you switch.

  • Vectors only, no keywords

    Semantic search blurs exact identifiers: order 40718829 and 40718892 look alike as vectors. Add keyword search for codes, SKUs and names, and fuse the rankings.

  • Sizing on raw vectors alone

    HNSW graphs add memory on top of the raw vectors, and index builds need headroom. Load-test with production-sized data before committing to an instance size.

  • Forgetting tenancy and deletion

    A shared index without strict tenant filters can leak one business's content into another's answers, and an erasure request must reach the vectors as well as the source text. Partition by tenant and delete both.

Example: One index for six brands in Mumbai

A Mumbai company runs WhatsApp support for six consumer brands from one platform, with about 400,000 knowledge-base chunks in English, Hindi and Marathi. Exact search grew too slow at peak, so the team built an HNSW index and measured recall@5 above 0.95 against exact search at its chosen ef_search, with P99 query latency inside its budget. Early on, a post-filter on tenant_id sometimes left the smallest brand with only one or two chunks per query; partitioning the index by tenant fixed it. Adding keyword search alongside vectors fixed the remaining misses on product codes.

Best practices for a vector database

  • Start with a vector index in a database you already run; move to a dedicated engine when scale, filtering or latency demand it.
  • Normalise vectors once at write time so cosine similarity and dot product rank results identically.
  • Keep source text and IDs outside the index as well, so you can rebuild it from scratch.
  • Turn on encryption at rest and back the index up — embeddings can reveal information about the text they encode.

Vector Database FAQs

What is the difference between a vector database and a knowledge base?+
A knowledge base is the content; a vector database is the infrastructure for searching it by meaning. You curate articles and policies in the knowledge base, embed them, and store the embeddings in the vector database for retrieval.
Is HNSW better than IVF?+
HNSW usually gives better recall at low latency but uses more memory and builds more slowly. IVF builds faster and uses less memory but needs `nlist` and `nprobe` tuned, and is often paired with compression for very large collections.
Do I need a dedicated vector database for a chatbot?+
Often not at first. Tens or hundreds of thousands of chunks run well on a vector index inside an existing database. Dedicated engines earn their place with very large collections, heavy filtering or strict latency targets.
Is vector search better than keyword search?+
Neither wins everywhere. Vectors match meaning and paraphrases, including mixed-language questions; keywords match exact codes, names and numbers. Hybrid search, which fuses both rankings, is the usual production choice.

See InfiQ pricing

Try it on the official WhatsApp Business API

InfiQ is an official Meta Business Partner. Start a 7-day free trial — no card needed — and see this running on your own number.

Start free trialBrowse all 550 terms