AI PDF Chatbot With LangChain 2026: Build One in 30 Minutes
A PDF chatbot is a four-step pipeline: load, split, embed, retrieve-and-answer. Here is complete runnable code, why each parameter is set that way, and how it breaks.
📜 Table of Contents
AI PDF Chatbot with LangChain 2026: Build One in 30 Minutes
Turning a PDF into a chatbot is four steps: load, split, embed, retrieve-and-answer. Under a hundred lines.
1. Dependencies
pip install langchain langchain-community langchain-chroma langchain-openai langchain-text-splitters pypdf
The model need not be OpenAI — DeepSeek, Qwen and local Ollama all expose OpenAI-compatible endpoints; change base_url and api_key.
2. The whole pipeline
docs = PyPDFLoader("handbook.pdf").load()
splitter = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=120, add_start_index=True)
chunks = splitter.split_documents(docs)
store = Chroma.from_documents(chunks, OpenAIEmbeddings(model="text-embedding-3-small"),
persist_directory="./chroma_db")
retriever = store.as_retriever(search_kwargs={"k": 4})
prompt = ChatPromptTemplate.from_messages([
("system", "Answer only from the context. If it is not there, say you do not know."),
("human", "Context:\n{context}\n\nQuestion: {question}"),
])
def ask(q):
hits = retriever.invoke(q)
ctx = "\n\n".join(f"[p{h.metadata.get('page', 0) + 1}] {h.page_content}" for h in hits)
chain = prompt | ChatOpenAI(model="gpt-4o-mini", temperature=0) | StrOutputParser()
return chain.invoke({"context": ctx, "question": q})
3. Why these parameters
| Parameter | Value | Reason |
|---|---|---|
chunk_size |
500–1000 chars | too small loses context, too large hurts precision |
chunk_overlap |
10–20% | stops sentences being cut in half |
k |
3–5 | more just adds noise and cost |
temperature |
0 | factual answers need determinism |
4. When answers are wrong
Print retriever.invoke(question) first. If the right passage is not retrieved, the problem is chunking or embeddings, not the model. Then check whether tables or multi-column layouts got shredded, whether the answer spans pages (use sentence-window retrieval), and whether the prompt forbids guessing. The first three fix most cases.
5. What to add next
Conversation memory, per-source collections, citations with page numbers, embedding cache, and finally a FastAPI wrapper. Build the API layer last — fix answer quality first.
6. Real pitfalls
Scanned PDFs have no text layer (check len(docs[0].page_content); if near zero, run OCR). Chinese splits poorly with default separators — use ["\n\n", "\n", "。", "！", "？", "；", "，", ""]. Tables lose structure; parse them with pdfplumber and store as Markdown. Indexing costs more than querying — size the embedding job before running it. And if you change embedding models, delete the old vector directory, or you will mix incompatible vector spaces.
FAQ
Chroma required? No — FAISS, Qdrant, pgvector all work behind as_retriever().
Multiple PDFs? Concatenate the loaded document lists; filenames stay in metadata for filtering.
Fully accurate? No. RAG makes answers traceable, not infallible. Citations are the real accuracy lever.
No OpenAI account? ChatOpenAI(base_url="http://localhost:11434/v1", api_key="ollama", model="qwen2.5:7b") plus OllamaEmbeddings.
Related
❓ FAQ
Why LangChain instead of building raw?
Components handle the fiddly parts (splitting, retrieval, prompt assembly); raw code is educational, LangChain is productive.
Which embedding model?
text-embedding-3-small is cheap and good enough; local users can use Ollama embeddings.
Does this work with scanned PDFs?
No - PyPDFLoader needs text; add OCR (pytesseract) for scans.
How accurate is it?
Depends on chunk quality and the LLM; the k=4 retrieval with source citations is the accuracy lever.
Written by our editorial team; tools listed here are tested or verified against public sources. Links point to official sites or GitHub repos for reference only — no paid placements.