Anthropic's Fable 5.2 model topped the Remote Labor Index (RLI) benchmark from Scale Labs and the Center for AI Safety, bringing 41.3% of real Upwork orders to a client-acceptable level—roughly double that of its nearest competitors, Opus 5.5 and GPT 6 Astra—while a year ago the best agent, Manus, completed only 2.5% of similar tasks. Progress is most noticeable in audio and music editing, architectural and interior design, and 3D modeling. However, agents still fail to complete more than half of real orders.

!image&subtitle=Scale%20Labs&theme=light)

image

What happened

Scale Labs and the Center for AI Safety updated the Remote Labor Index leaderboard—a test where agents perform real commercial orders from Upwork rather than synthetic task sets. Anthropic's Fable 5.2 model took first place: 41.3% of orders were accepted as suitable for client delivery. The scores for Opus 5.5 (21.25%) and GPT 6 Astra (20.83%) were roughly half as high, the previous version Fable 5.1 scored 17.9%, and the best agent a year ago, Manus, managed only 2.5% of tasks. Fable 5.2's main gains are concentrated in audio and music editing, architectural and interior design, and 3D modeling: for the first time in the benchmark's history, the model solved nearly 50 tasks that had defeated every system. In industrial design and animation, results remain amateurish or produce broken files.

Context

The Remote Labor Index is structured differently from familiar tests on isolated skills: it measures an agent's ability to see a real commercial job through to completion. The sample includes 240 projects from 358 verified freelancers with a total value of approximately $144,000; the average order costs $632.60 and takes about 29 hours of work. Key metrics here are economic: cost, execution time, and the share of orders accepted by the client. The benchmark keeps result verification manual—experts open final files in Blender, GIMP, and Audacity because AI reviewers cannot handle professional software. It is precisely thanks to this honest acceptance that the capability growth achieved by agents is difficult to attribute to measurement noise or marketing effects.

Why this matters for the industry

For the industry, RLI sets a publicly available baseline that previously barely existed: the benchmark measures end-to-end commercial work with economic metrics—cost, time, and the share of accepted work—rather than individual skills on hypothetical tasks. The jump from 2.5% to 41.3% in a year shows that agents are beginning to close entire categories of freelance work: audio and music editing, architectural and interior design, and 3D modeling. This directly affects the remote work market, freelance service pricing, and the strategies of companies automating production processes. Even now, in the benchmark's strong categories, it is possible to assemble pilot pipelines of the "brief — agent — human review — delivery" type and measure their benefit by the share of accepted work. Competition between Fable 5.2, Opus 5.5, and GPT 6 Astra turns this benchmark into a subject of direct racing: labs will likely start deliberately strengthening weak categories, and the leaderboard will change rapidly.

Why this matters for users

For readers, this is a specific indicator of which tasks AI already really performs "for client delivery" and where the result is still amateurish or broken. If you order or perform work in audio editing, interior design, or 3D modeling, agent drafts and semi-automatic delivery will appear in these categories earlier than anywhere else. Industrial design and animation, conversely, remain a zone where the model outputs broken source files, so human skills here are currently unmatched. The RLI leaderboard is open: one can independently monitor model dynamics and timely decide which skills to study and which are already partially automated.

What is still unknown / limitations

Conclusions about the trajectory are still shaky: optimistic growth scenarios of "faster than linear" rely on only three data points—2.5% for Manus, 17.9% for Fable 5.1, and 41.3% for Fable 5.2—with a sample of 240 projects, and three measurements do not yet form a curve. The 41.3% result means that agents still do not complete more than half of real orders, so the effect remains measurable rather than market-driven. Data for production use is lacking: published materials do not include inference cost, latency, or reliability metrics for repeated runs, and the jump within the Fable line requires separate verification to separate the model's contribution from prompt engineering. Manual acceptance of work by experts reduces systematic error from AI judges but adds subjectivity and does not scale well, so benchmark results are not a guarantee of quality in real client work.

Sources

Author

Look at AI, editorial team