RAG Cost per 1,000 Questions: $0.54 to $1.53, Measured

📘 Tutorials 2026-11-19 6 min read

RAG spend estimates are built from document counts, the index looks expensive, and nobody can say whether retrieving four passages instead of two is worth the difference.

📜 Table of Contents

Everyone budgets the index and forgets the context window. On a three-page documentation corpus the entire embedding pass costs 43 hundredths of a cent, and one thousand answered questions cost between $0.54 and $1.53 depending on who generates the answer. The index is 0.08% of the first bill. If you are estimating RAG spend from the number of documents, you are measuring the wrong side.

RAG costs $0.54 to $1.53 per 1,000 questions while embedding the whole corpus costs $0.00043 at 0.08% of the bill
21,499 tokens across three documents; at k=4 each question sends 2,198 input tokens and returns 350 output tokens. Per 1,000 questions: $0.540 on deepseek-flash off-peak, $1.079 peak, $1.250 on gpt-5-mini, $1.534 on gemini-2.5-flash, a 2.84x spread on identical context. Embedding the entire corpus costs $0.00043, or 0.08% of the bill. Computed 19 November 2026.

What was measured, and how

Three real documentation pages, fetched 19 November 2026: the LiteLLM proxy configuration guide, the vLLM optimization guide, and the RAGFlow README. Each was stripped of scripts and tags, then tokenised with tiktoken's cl100k_base encoding - the same tokeniser family used by the GPT-3.5/4 generation of models, and close enough to DeepSeek's and Gemini's for a planning number. Chunking was 512 tokens with 64 tokens of overlap.

pagechars of texttokens512-token chunks
LiteLLM proxy configs35,1928,84220
vLLM optimization guide38,0088,18019
RAGFlow README16,9324,47710
total90,13221,49949

Two numbers worth keeping: 4.19 characters per token, and an average of 7,166 tokens per page of technical documentation. A 100-page internal wiki at that density is about 716,000 tokens of text. The text-per-token ratio is lower than the usual "four characters" rule because API docs are dense with code and identifiers, which tokenise badly.

Line one of the bill: the index

Embedding 21,499 tokens with a model priced at $0.02 per million input tokens (text-embedding-3-small, per LiteLLM's published price map, 19 Nov 2026) costs $0.00043. The 100-page wiki: $0.0143. Re-embedding the whole corpus every night for a year is under four dollars at that rate, which is why "vector database costs" is almost never the line item that hurts. The same is true of a self-hosted vector store: pgvector and Qdrant both have zero marginal cost per vector at this scale, and licensing is not the constraint.

Line two: the context you hand the model

Retrieval is where the money moves. Ask a question, pull the top-k chunks, and every chunk is input tokens you pay for on every single question, forever, even though the corpus is paid for once. With k=4 that is 2,048 tokens of passages; add a 150-token block of instructions plus the question and you are at 2,198 input tokens per question. A 350-token answer is 350 output tokens.

Prices below are per million tokens, checked 19 November 2026: DeepSeek's own pricing page for deepseek-flash (cache-miss input $0.15 off-peak, $0.30 peak; output $0.60 off-peak, $1.20 peak), and LiteLLM's price map for gemini-2.5-flash ($0.30 / $2.50) and gpt-5-mini ($0.25 / $2.00). LiteLLM's map lags vendor pages, so treat those two as planning figures and confirm on the provider's page before you commit.

modelinput, per 1,000 qoutput, per 1,000 qtotal
deepseek-flash, off-peak$0.330$0.210$0.540
deepseek-flash, peak$0.659$0.420$1.079
gpt-5-mini$0.549$0.700$1.250
gemini-2.5-flash$0.659$0.875$1.534

Ten thousand questions a month - roughly 330 a day - lands at $5.40 to $15.34. Someone deciding whether to build the thing should notice that this is not a budget item at all; the reason to care about the spread is that it is 2.8 times, and the same factor applies at a million questions a month, where it becomes $540 against $1,534.

Lever one: how many chunks you retrieve

passages keptinput tokens per questionper 1,000 q, deepseek-flash off-peakgemini-2.5-flash
21,174$0.386$1.227
31,686$0.463$1.381
42,198$0.540$1.534
63,222$0.693$1.842

Here is the part that contradicts the usual advice. Going from four passages to two saves $0.154 per thousand questions. At 100,000 questions a month that is $15. If retrieval quality is the thing standing between a useful answer and a wrong one - and in our own test on sixty questions, giving the model four retrieved passages moved correct answers from 21 to 47 - then cutting k in half to save $15 a month is a bad trade. Cost the knob before you turn it, then leave it where quality wants it.

Lever two: answer length

Output tokens cost between four and eight times input tokens, so the length of the answer is the largest single dial in the bill. Same 2,198-token input, three answer lengths:

answer tokensdeepseek-flash off-peakgemini-2.5-flash
150$0.420$1.034
200$0.450$1.159
350$0.540$1.534
500$0.630$1.909

Asking for a 200-token answer instead of 350 cuts 16.7% off the DeepSeek bill and 24.4% off Gemini's. That is a bigger saving than halving the number of passages, and unlike the passage count it rarely makes the answer worse - long answers are usually padding around the same two sentences. Instructing "answer in under 150 words, cite the passage" is the cheapest change available.

Lever three: the cached prefix

The instruction block in front of every question is identical text. DeepSeek bills cache hits at $0.003 per million against $0.15 cache-miss, a 50x difference. Put the static part first, unchanged, and cache 1,000 of the 2,198 input tokens: the thousand-question bill drops from $0.540 to $0.393, a 27.2% cut, and the same 27.2% at peak rates. The condition is that the prefix is byte-identical and comes first; one changing timestamp in the system prompt and nothing is cached, silently.

Reproducing the measurement

Do this on your own corpus rather than trusting the numbers above; the spread between a prose wiki and an API reference is large enough to change the answer.

import tiktoken, re
enc = tiktoken.get_encoding("cl100k_base")
text = open("doc.md", encoding="utf-8").read()
ids = enc.encode(text)
step, chunks, i = 512 - 64, 0, 0
while i < len(ids):
    chunks += 1
    i += step
print(len(text), len(ids), round(len(text)/len(ids), 2), chunks)

Then multiply: tokens/1e6 * embedding_price for line one, and (chunks_kept * chunk_size + instructions) * questions * input_price + answer_tokens * questions * output_price for line two. Everything else on a RAG invoice is a rounding error next to those two.

What this does not cover

No latency, no quality comparison between the four models, and no reranker cost - a reranker is billed per search rather than per token at most vendors, so it does not fit this arithmetic and was excluded rather than estimated. The token counts came from a tokeniser that is not identical to DeepSeek's or Gemini's; expect a few percent either way, which does not change any decision above. For the document-parsing side of the pipeline, see the parsing comparison, and for evaluating whether the answers are any good before you optimise their price, see the RAGAS harness walkthrough.

Related Articles
2026-07-19
2026 AI Coding Tools Comparison: Cursor vs Aider – Which One Wins?
2026-10-24
Ragas 0.4 Tutorial: An Evaluation Harness That Splits Retrieval Failures from Generation Failures
2026-07-18
Hybrid AI Coding: When to Use Claude Code Cloud vs Local Models
2026-08-05
How to Build a Discord AI Bot in 2026: discord.py (16k Stars) + LLM - Step by Step
2026-08-11
Local LLM Hardware Guide 2026: How Much RAM and VRAM You Really Need
2026-07-26
Local LLM Hardware Requirements

Written by our editorial team; tools listed here are tested or verified against public sources. Links point to official sites or GitHub repos for reference only — no paid placements.

💬 Comments (0)

No comments yet. Be the first!

Login to comment