LLM Evaluation Benchmark Rankings: Which Model Tops the Charts

LLM Evaluation Benchmark Rankings: Which Model Tops the Charts

💡 What You Will Learn

LLM Evaluation Benchmark Rankings: Which Model Tops the Charts

|:----|:--------|:---:|:--------| | MMLU-Pro || 12K || | HumanEval || 164 || | GSM8K || 8.5K || | C-Eval || 9K || ||| MMLU-Pro | HumanEval | GSM8K | Arena Elo | |:---:|:----|:-------:|:--------:|:----:|:--------:| | 1 | GPT-4o | 88.3 | 92.1 | 95.5 | 1523 | | 2 | Claude 4 Sonnet | 87.6 | 90.4 | 94.2 | 1498 | | 3 | DeepSeek R1 | 85.8 | 88.7 | 96.1 | 1476 | | 4 | Qwen3-72B | 84.2 | 87.3 | 93.8 | 1462 | | 5 | Gemini 2.5 Pro | 86.1 | 89.5 | 94.7 | 1485 |

pip install lm_eval

# 
lm_eval --model hf \
  --model_args pretrained=Qwen/Qwen2.5-7B \
  --tasks mmlu_pro,gsm8k,hellaswag \
  --device cuda:0 \
  --batch_size auto \
  --output_path ./results

# 
lm_eval --model openai-completions \
  --model_args model=gpt-4o \
  --tasks mmlu_pro,gsm8k \
  --output_path ./results

import json
with open("./results/results.json") as f:
    results = json.load(f)
for task, scores in results["results"].items():
    acc = scores["acc"] * 100
    print(f"{task}: {acc:.1f}%")