Vector databases power RAG systems, similarity search, and AI memory. But what are they? How do they work? How do you choose one?

💡 What You Will Learn

Vector databases power RAG systems, similarity search, and AI memory. But what are they? How do they work? How do you choose one?

📜 Table of Contents

• What Is a Vector?
• How Vector Databases Work
• Popular Vector Databases
• Using Vector DBs with LLMs (RAG)
• Key Concepts
• Use Cases Beyond RAG
• FAQ

Vector Database Beginner Guide: What They Are and How to Use Them with LLMs

A vector database stores and searches data as mathematical vectors (embeddings). Unlike traditional databases that search by exact match, vector databases find the "most similar" items.

What Is a Vector?

A vector is a list of numbers that represents the meaning of data:

"cat" → [0.23, 0.87, -0.12, 0.45, 0.91, ...]  # 384 numbers
"dog" → [0.25, 0.85, -0.10, 0.42, 0.88, ...]  # Similar to cat
"car" → [-0.31, 0.12, 0.76, -0.54, -0.08, ...] # Different from cat

How Vector Databases Work

Text → Embedding Model → Vector → Store → Query → Similarity Search → Results

Popular Vector Databases

pip install chromadb

import chromadb

client = chromadb.PersistentClient(path="./my_vectors")
collection = client.create_collection(name="my_documents")

collection.add(
    documents=["LangChain is a framework for LLM apps.",
               "RAG stands for Retrieval Augmented Generation."],
    ids=["doc1", "doc2"]
)

results = collection.query(query_texts=["What is RAG?"], n_results=2)
print(results["documents"][0])

Using Vector DBs with LLMs (RAG)

from langchain_community.vectorstores import Chroma
from langchain_community.embeddings import OllamaEmbeddings
from langchain_community.llms import Ollama
from langchain.chains import RetrievalQA

vectorstore = Chroma(
    collection_name="my_documents",
    embedding_function=OllamaEmbeddings(model="nomic-embed-text"),
    persist_directory="./my_vectors"
)

qa_chain = RetrievalQA.from_chain_type(
    llm=Ollama(model="llama3.1:8b"),
    retriever=vectorstore.as_retriever(search_kwargs={"k": 3})
)
result = qa_chain.invoke({"query": "What is RAG?"})
print(result["result"])

Key Concepts

• Cosine Similarity: Best for text (measures angle between vectors)
• HNSW: Fast approximate search (ChromaDB default)
• Chunk Size: 512 tokens with 50 overlap is recommended

Use Cases Beyond RAG

1. Semantic search, 2. Recommender systems, 3. Anomaly detection, 4. Deduplication

FAQ

Q: Do I need a GPU? A: No. Embedding works on CPU. Q: How many vectors on a laptop? A: ChromaDB handles 10M+ on 16GB RAM. Q: Best embedding model? A: BGE-small-en-v1.5 (speed) or BGE-base-en-v1.5 (quality).

❓ FAQ

How many vectors on a laptop?

ChromaDB handles 10M+ on 16GB RAM.

Best embedding model?

BGE-small-en-v1.5 (speed) or BGE-base-en-v1.5 (quality).