JobBench

JobBench is a test, published in May 2026, that measures how well AI agents (software that carries out a task on its own) handle the work professionals most want to hand off, such as a reporter's fact-checking across sources. It covers 130 tasks from 35 occupations. At launch the best agent scored 45.9%, so on this kind of work an agent's output still needs checking.

Where do the tasks come from?

According to the authors, current benchmarks for work agents, such as OpenAI's GDPval (2025), choose tasks mainly by economic value. JobBench, by researchers from the University of Washington and other institutions (Li et al.), starts from what workers say they want to delegate. Its basis is WorkBank (Shao et al., 2025), a survey in which 1,500 workers rated, duty by duty, whether they would like an AI agent to take over parts of their own job.

The authors picked 35 occupations with an average wish to delegate above 3 on a 5-point scale and high total wages. They include reporters, customer service representatives, bookkeeping clerks, online merchants and lawyers. Experts recruited through Prolific and Upwork drafted the tasks, which were expanded with AI help and had to pass three checks. The result is a main set of 65 tasks and an easier set of 65.

What does a task look like?

Each task is a folder with an instruction, the deliverables to produce, and reference files: 3.9 on average and up to 15, in formats such as CSV, PDF and Excel. The files can contradict each other, and the agent has to find and reconcile the right sources. In the paper's reporter example, the agent cross-checks water-quality data, EPA guidance and health surveillance data across years before it assembles an editorial package. In the main set, 51.7% of the files come from real public records; the rest are synthetic.

How is it scored?

Each task has chains of yes-or-no checks, 35.6 checks per task on average and 4,631 in total, each tied to a number, a reasoning step or a documented professional judgement. A chain only earns points when every check in it passes, so the right fact reached by a wrong route earns nothing. A language model does the grading: Grok 4.1 Fast by default, checked against Claude Opus 4.5, with scores within 0.7 points of each other.

What were the results?

The authors tested 36 combinations of model and agent software (Claude Code, Codex CLI, OpenCode and OpenClaw). On the main set, Claude Opus 4.7 in Claude Code scored 45.9%, GPT-5.5 in Codex CLI 42.7% and GPT-5.4 in Codex CLI 38.9%. Outside the Claude and GPT families, no configuration went above 19%. On GDPval, GPT-5.4 matches or beats expert work in 83.0% of comparisons, which the authors call close to saturation. The agent software matters as well: Claude Sonnet 4.6 scored 36.9% in Claude Code and 30.6% in OpenClaw. A full run of the main set cost between $32 and $210 at list prices. By September 2026 the project's leaderboard showed a higher top score: 65.7% for Claude Opus 5.

What does it not show?

The scores describe the models and agent software of spring 2026. The occupations and duties come from the American O*NET database, and a language model does the grading. JobBench is one approach to agent evaluation; the paper names GDPval, the Remote Labor Index and $OneMillion-Bench as others. The evaluation code is public on GitHub.