Agentsagentsmultimodalknowledge-workdocuments

Multimodal Knowledge Agent (MKA)

MKA evaluates agents on visually complex knowledge-work tasks involving documents, spreadsheets, reports, and other files where layout and visual information matter.

Last reviewed: 19 November 2026

What it measures

Whether an agent can reason across visually complex files and produce deliverables for real-world knowledge-work tasks, not just answer text-only questions.

Input / Output

Input: visually complex files and a task. Output: a completed deliverable. Metric: task quality scored against an expert rubric.

Evaluation methodology

Artificial Analysis runs a private 200-task set in the open-source Stirrup agent harness with the same tools, sandbox, and turn budget for each model. A non-Google agentic autorater grades deliverables; Google reports 96% agreement with expert human grading. Ten public tasks are released for format inspection, but the full task set is private.

Metadata

Task
Multimodal knowledge-work agent tasks
Domain
Documents, spreadsheets, reports, and visual knowledge work
Modality
Text + visually complex files
Input type
Files + task instruction
Output type
Completed knowledge-work deliverable
Evaluation type
Rubric-based agentic grading
Primary metric
Task quality score
Secondary metrics
Cost per task, Average turns, Human/autorater agreement
Paper
MKA: Evaluating Multimodal Knowledge Agents on end-to-end professional tasks
GitHub
Stirrup agent harness
Dataset
MKA benchmark announcement and methodology
Leaderboard
Official live leaderboard (external)

Live leaderboard

Rankings change as new evaluations are published. View current results on the official leaderboard.

View live leaderboard →

Datasets

GitHub repositories

Related research

Related guides

Related tools

Related models

Companies

Related rankings

Explore more benchmarks

All benchmarks →