Agentsagentsmultimodalknowledge-workdocuments
Multimodal Knowledge Agent (MKA)
MKA evaluates agents on visually complex knowledge-work tasks involving documents, spreadsheets, reports, and other files where layout and visual information matter.
Last reviewed: 19 November 2026
What it measures
Whether an agent can reason across visually complex files and produce deliverables for real-world knowledge-work tasks, not just answer text-only questions.
Input / Output
Input: visually complex files and a task. Output: a completed deliverable. Metric: task quality scored against an expert rubric.
Evaluation methodology
Artificial Analysis runs a private 200-task set in the open-source Stirrup agent harness with the same tools, sandbox, and turn budget for each model. A non-Google agentic autorater grades deliverables; Google reports 96% agreement with expert human grading. Ten public tasks are released for format inspection, but the full task set is private.
Metadata
- Task
- Multimodal knowledge-work agent tasks
- Domain
- Documents, spreadsheets, reports, and visual knowledge work
- Modality
- Text + visually complex files
- Input type
- Files + task instruction
- Output type
- Completed knowledge-work deliverable
- Evaluation type
- Rubric-based agentic grading
- Primary metric
- Task quality score
- Secondary metrics
- Cost per task, Average turns, Human/autorater agreement
- Paper
- MKA: Evaluating Multimodal Knowledge Agents on end-to-end professional tasks
- GitHub
- Stirrup agent harness
- Dataset
- MKA benchmark announcement and methodology
- Leaderboard
- Official live leaderboard (external)
Live leaderboard
Rankings change as new evaluations are published. View current results on the official leaderboard.
View live leaderboard →Datasets
GitHub repositories
Related research
Related guides
Related tools
Related models
- GPT-5
OpenAI
- Claude Opus
Anthropic
- Gemini 3.1 Pro
Google
- DeepSeek R1
DeepSeek
- Qwen3
Alibaba
- Llama 4
Meta
Companies
Related rankings
Explore more benchmarks
- GAIAAgentsGeneral AI Assistants benchmark: real-world questions requiring tool use, multi-step reasoning, and web/file interaction — hard for both humans and agents.
- WebArenaAgentsRealistic web environment benchmark where agents must complete tasks on self-hosted sites (e.g. e-commerce, CMS, forums) via browser actions.
- MMMUMultimodalMassive Multi-discipline Multimodal Understanding: college-level problems combining images and text across six disciplines.
- Terminal-BenchCodingTerminal-Bench evaluates agents on end-to-end terminal tasks in reproducible container environments. Version 4.1 hardens task verifiers against reward hacking and regrades stored 4.0 artifacts under the updated verifiers.
- LiveBenchGeneralLiveBench 2.0 evaluates long-horizon AI work across agentic tasks, design, coding, browser use, data analysis, research, and reasoning.
- SWE-BenchCodingEvaluates systems on real GitHub issues: given a repository and issue, produce a patch that passes the project’s tests.