RAGragcorrectiveretrievalfactuality
CRAG
Comprehensive RAG benchmark from Meta stressing retrieval failures, outdated knowledge, and answer faithfulness across realistic industry scenarios.
Last reviewed: 23 July 2026
What it measures
How RAG systems behave when retrieval is incomplete, noisy, or conflicting — not only when the right passage is easy to find.
Input / Output
Input: queries over corpora with controlled retrieval conditions. Output: answers + citations; scored for correctness and groundedness.
Evaluation methodology
Multi-scenario evaluation (accurate retrieval, incomplete, incorrect). Report scenario-level and aggregate scores.
Metadata
- Task
- Realistic / corrective RAG
- Domain
- Industry QA / factuality
- Modality
- Text
- Input type
- Query + retrieval conditions
- Output type
- Answer (+ optional citations)
- Evaluation type
- Scenario-based automatic scoring
- Primary metric
- Accuracy / score by scenario
- Secondary metrics
- Dynamic vs static, Incomplete retrieval score
- Paper
- CRAG — Comprehensive RAG Benchmark
- GitHub
- facebookresearch/CRAG
- Dataset
- facebookresearch/CRAG
- Leaderboard
- Official live leaderboard (external)
Live leaderboard
Rankings change as new evaluations are published. View current results on the official leaderboard.
View live leaderboard →Datasets
GitHub repositories
Related research
Related guides
Related tools
Related models
Companies
Related rankings
Explore more benchmarks
- RAGBenchRAGBenchmark suite for retrieval-augmented generation systems — evaluating retrieval quality, groundedness, and answer correctness on realistic RAG workloads.
- LongBenchLong ContextBilingual long-context benchmark covering single/multi-document QA, summarization, few-shot learning, synthetic tasks, and code across long inputs.
- GAIAAgentsGeneral AI Assistants benchmark: real-world questions requiring tool use, multi-step reasoning, and web/file interaction — hard for both humans and agents.
- HELMHolisticHolistic Evaluation of Language Models (Stanford CRFM): multi-metric, multi-scenario evaluation emphasizing transparency and taxonomy of use cases.
- Arena HardChatChallenging Chatbot Arena prompts packaged for automatic evaluation — a harder preference-style benchmark derived from real user battles.
- GPQAScienceGraduate-Level Google-Proof Q&A: PhD-level science questions designed so that non-experts with web search still struggle, testing deep domain reasoning.