All articles
5 min readai / productivity

Agents finished 30.3% of real office tasks: how to split the work

Teams I train ask the same question early: can an agent own this workflow from start to finish? Four benchmarks have measured that on realistic work, and the ceiling sits between 12% and 31%. On 175 tasks inside a simulated software company, the best agent finished 30.3% of them 1. The planning assumption I keep hearing in the room is 80%.

What four benchmarks found.

TheAgentCompany reproduced a small software firm: a GitLab server, an internal wiki, a chat tool staffed by simulated colleagues, and 175 tasks spread across software engineering, project management, data science, finance, HR and administration. Gemini 2.5 Pro completed 30.3% of the tasks outright and scored 39.3% on a metric that pays partial credit for intermediate checkpoints 1. Claude 3.7 Sonnet reached 26.3% and 36.4% in the same harness. The Gemini run averaged 27.2 steps and $4.2 per task.

WebArena set 812 tasks on self-hosted copies of an online store, a forum, a GitLab and a CMS. Its best GPT-4 agent finished 14.41%. People given the same tasks finished 78.24% 2. OSWorld sat agents in front of a working desktop with 369 tasks covering file management, office documents and workflows that cross several applications: humans 72.4%, the best agents under 12.2% 3.

Those levels age fast. METR measures the length of software task a model completes with 50% success, and found that horizon doubling roughly every seven months since 2019 (212 days, 95% CI 171 to 249 days) 4. Read the trend and discount the level, because these numbers shift under you within months. One shape in that paper does hold: horizons measured at an 80% success rate run far shorter than the same model's 50% horizon. The work a model handles dependably is much smaller than the work it handles half the time.

Right once, wrong by the eighth run.

Tau-bench scores agents with pass^k, where a task counts only if the agent solves it on all k independent attempts. GPT-4o solved about 61% of the retail tasks on a single attempt. Across 8 attempts, the share it got right every time dropped to about 25% 5. On the airline domain it managed 35.2% on one attempt.

That gap decides whether a workflow runs unattended. Chain the arithmetic: a 12-step process where each step lands 95% of the time completes 54% of the time. At 90% per step it completes 28%. A demo shows the per-step number. Production gives you the product of all of them.

The failures sit in the brief and in the checking.

Cemri and colleagues hand-annotated 150 execution traces from 7 agent frameworks and built MAST, a taxonomy of 14 failure modes in three groups: specification issues, misalignment between agents, and task verification 6. Agreement between annotators was high (kappa = 0.88), and the released dataset covers 1,642 traces. Two of their findings changed how I set agents up. Verification failures turn up in runs the system reported as successful, so an agent's own sign-off carries little information. And in several cases a single agent beat the multi-agent version built on the same model.

Kapoor and colleagues add the cost side. Their plain baselines, retry and warming and escalation, matched or beat several complex published agents on HumanEval at far lower cost, and they found agent benchmarks shipping with weak holdout sets or none at all 7. A leaderboard gain can come from paying for more attempts rather than from a better agent.

The protocol.

  1. Size each handoff under an hour of human work. Frontier models sit near their 50% horizon at that length, and their 80% horizon is shorter 4. A task you would hand a new colleague for the afternoon is too long for one agent run.
  2. Write the checkpoints before the prompt. Three to six per task, each a binary check a stranger could run: the file exists at this path, the test suite passes, the row count matches the source. TheAgentCompany had to score partial credit this way because whole-task completion stays rare 1. Track your own work at the same grain.
  3. Run the task 3 times on the same input. Promote it to unattended only when all 3 runs clear every checkpoint. One success proves the task is possible. Three identical successes suggest it is routine 5.
  4. Move the check outside the agent. A script, a second system or a person does the verifying. An agent reviewing its own output signs off on broken work 6.
  5. Beat the simple baseline first. One agent, one model, retry on failure. Add a second agent when it wins on your tasks, not on someone's benchmark 67.
  6. Price per completed task. The Gemini run cost $4.2 per attempt at a 30.3% completion rate, so roughly $14 per finished task 1. Budget on that second figure.
  7. Seal 20 tasks from your own backlog. Tune prompts and scaffolding on everything else, then measure on the held-back 20 and leave them untouched 7.

Start with step 2. Pick one workflow you want to hand off, write its checkpoints tonight, and run the agent against them 3 times tomorrow. The pass count across those 3 runs tells you more about that workflow than any leaderboard.

Sources.

  1. Xu, F. F., et al. (2025). TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks. NeurIPS 2025 Datasets and Benchmarks Track. arXiv:2412.14161
  2. Zhou, S., Xu, F. F., Zhu, H., et al. (2024). WebArena: A Realistic Web Environment for Building Autonomous Agents. ICLR 2024. arXiv:2307.13854
  3. Xie, T., et al. (2024). OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. NeurIPS 2024 Datasets and Benchmarks Track. arXiv:2404.07972
  4. Kwa, T., West, B., Becker, J., et al. (2025). Measuring AI Ability to Complete Long Software Tasks. METR. arXiv:2503.14499
  5. Yao, S., Shinn, N., Razavi, P., Narasimhan, K. (2024). tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. arXiv:2406.12045
  6. Cemri, M., Pan, M. Z., Yang, S., et al. (2025). Why Do Multi-Agent LLM Systems Fail? NeurIPS 2025 Datasets and Benchmarks Track. arXiv:2503.13657
  7. Kapoor, S., Stroebl, B., Siegel, Z. S., Nadgir, N., Narayanan, A. (2024). AI Agents That Matter. Transactions on Machine Learning Research (2025). arXiv:2407.01502

Keep reading