RAG评估指标2026：用Ragas衡量检索质量

RAG应用答得自信，但一半引用是错的。需要指标而不是感觉。

RAG Evaluation Metrics: The Numbers That Tell You Retrieval Is Broken

Most RAG failures are retrieval failures in disguise. The generator answers fine; the retriever just handed it the wrong documents. Ragas is the open-source framework built to measure this split, and with 15,076 GitHub stars (August 2026) it is the most widely used RAG evaluation library in the ecosystem.

The Four Metrics That Matter

Ragas defines four core metrics that map to the two failure modes:

Retrieval side: - Context Precision - of the chunks the retriever returned, how many were actually relevant to the question. - Context Recall - of all relevant chunks that exist in the corpus, how many did the retriever find.

Generation side: - Faithfulness - does the answer contain claims not supported by the retrieved context? Low faithfulness means hallucination. - Answer Relevancy - does the answer actually address the question asked, or did it drift to something adjacent?

A Concrete Workflow

from ragas import evaluate
from ragas.metrics import faithfulness, context_recall

result = evaluate(dataset, metrics=[faithfulness, context_recall])

Feed it question-answer-context triples, and Ragas uses an LLM judge to score each. Run this on every corpus change. If context recall drops from 0.82 to 0.61 after you switched embedding models, you caught the regression in ten minutes instead of discovering it from user complaints.

Where Teams Get Stuck

The most common mistake is evaluating only faithfulness. A RAG system can score 0.9 faithfulness while retrieving the same two chunks for every question - technically faithful, practically useless. Always pair a retrieval metric with a generation metric. If you only track one number, make it context recall, because it degrades first and predicts the other metrics falling a week later.

FAQ

Does Ragas need an LLM API? It uses an LLM as judge, so yes, but it works with any provider including local models.

How many samples do I need? 50-100 representative questions give a stable signal; 20 works for a first pass.

Is it only for RAG? It also evaluates agent tool-use traces and summarization tasks.

Ragas vs plain pytest? Ragas is purpose-built for RAG dimensions; pytest is the runner underneath.