ApprenticeBench: AI Tested as a New Employee

NeoCognition Lab released ApprenticeBench — the first end-to-end benchmark where an AI agent must learn the job of an accounts payable clerk in the Odoo ERP across 100 sequential tasks, with Claude Fable 5.1 solving 72% of them, outperforming the best of two human testers at 51%.

Compiled by Sergey KostenchukPublished 2026-09-17Updated 2026-09-18

2026-09-17 Research Anthropic

🤖 ApprenticeBench: AI Tested as a New Employee

NeoCognition Lab released the first end-to-end agent test on a real job: AI learns the role of an AP clerk in the Odoo ERP — 100 sequential tasks with a six-month archive of invoices, with mentoring gradually fading. Claude Fable 5.1 solved 72% of the tasks, GPT-6 Astra — 68%, the best human — 51%, and the open Kimi K3 — 18%.

🌍 For the first time, the full 'hiring' cycle of AI is measured: training on company data, working in a GUI without APIs, and adapting to changing policies. Only two flagships passed the 'new employee' threshold; the authors called the gap between closed and open models the largest they have seen.

👤 The leaderboard with prices is open at apprenticebench.com. The 'computer tax' disappeared only for two flagships, but the agent is still more expensive than a human: $18.23 per task versus $7.21, via API — $6.95.

Source 1: https://neocognition.io/blog/apprentice-bench/ Source 2: https://apprenticebench.com/

Sources