LLM Evaluation Metrics Papers in 2026: 8 Foundational Papers to Read

Eight papers that defined how we evaluate LLMs, from MMLU to RAGAS.

## LLM Evaluation Metrics Papers in 2026 If you want to understand why we measure LLMs the way we do, these 8 papers are the foundation. All are publicly available on arXiv. ### The essential list 1. **MMLU (2020)** - Measuring Massive Multitask Language Understanding. Hendrycks et al. Defined the 57-subject benchmark that still dominates general evaluation. 2. **HELM (2022)** - Holistic Evaluation of Language Models. Stanford team; introduced multi-metric, multi-scenario evaluation. 3. **GSM8K (2021)** - Training Verifiers to Solve Math Word Problems. OpenAI; the standard arithmetic reasoning benchmark. 4. **HumanEval (2021)** - Evaluating Large Language Models Trained on Code. OpenAI; pass@k code generation. 5. **Chain-of-Thought (2022)** - Wei et al. Showed that prompting for reasoning steps improves accuracy on GSM8K, MATH, and BBH. 6. **RAGAS (2023)** - Automated Evaluation of Retrieval Augmented Generation. Defined faithfulness, answer relevancy, and context precision. 7. **MT-Bench (2023)** - Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. Introduced LLM-as-judge evaluation. 8. **LLM-as-a-Judge (2023)** - Zheng et al. Validated using GPT-4 as an evaluator, now standard practice. ### How these shape 2026 practice - MMLU + HumanEval + GSM8K remain the default combo for model cards - RAGAS is the default for RAG systems - LLM-as-judge replaced most human eval in CI pipelines ### FAQ **Where can I read them?** All are free on arXiv.org. **Do I need to read all eight?** Start with MMLU, RAGAS, and LLM-as-a-Judge - they cover 80% of daily practice.