Poobesh Gowtham published arc-skill — a skill for Claude Code on Claude Opus 5 that solved all 25 public ARC-AGI-3 games with a ceiling score of 100.00 RHAE, and the result was officially verified by the ARC Prize Foundation. The model was not fine-tuned: the result came from a skill that turns each agent action into a logged, verifiable experiment.

image
image

What happened

The skill consists of a 129-line SKILL.md instruction file and a 4343-line CLI tool, in which none of the 25 games is mentioned even once. The mechanics reduce to a single rule: before each action, the model formulates a falsifiable prediction of what will change on the grid — in one of 8 forms, e.g., cell X,Y=V, move, level+1 — and the harness rejects actions without a prediction and checks it against the actual returned frame. Over the campaign, the model completed 183 of 183 levels in 7645 actions and wrote 7627 predictions, 443 of which missed. The official record of the result is scorecard 24ddb219, published on August 13, 2026; all button presses were replayed through ARC servers.

Context

The public slice of ARC-AGI-3 consists of 25 games, the score is measured in RHAE, and the human median on this slice is 17135 actions. Against this backdrop, the key fact: without the skill, the same Claude Opus 5 scores only 30.16% on the same benchmark, so the gap to the scale ceiling was closed by the harness, not the model. The run does not look like "got it on the first try": single test actions were wrong 37.1% of the time, while actions within a formed plan were wrong only 2.9%, meaning the agent learned the game rules from its own mistakes, and the success rests on an iteration budget. In 24 of the 25 games, the agent used offline Python for calculations (1727 calls), so a significant part of the result is the model working in code, not pure visual reasoning.

Why this matters for the industry

For the industry, this is a verified, reproducible example of skill engineering — packaging tool use and a protocol into a skill file — as a practical test-time alternative to fine-tuning and prompt engineering for tasks with strict output formats: the cost of the skill and harness is many times lower than the cost of retraining the model. At the same time, the public ARC-AGI-3 leaderboard has effectively stopped ranking agents: after a verified ceiling on the public slice, meaningful comparison is possible only on private tasks, and benchmark evaluation will likely shift to closed or rotating sets.

Why this matters for users

The predict-before-act pattern can be transferred to your own agent without changing the model: add a step to the agent loop where the model records a prediction before acting, then receives automatic verification against the actual result and an error log — this turns the agent's wandering into an experiment with a verifiable outcome. The arc-skill itself is installed with a single command, npx skills add pbshgthm/arc-skill; ARC_API_KEY and Python 3.12+ are required. The harness's prediction and miss log is a ready-made dataset for evals and observability; a full breakdown with transcripts of all runs is at arc-skill.vercel.app, and verified replays of all 25 games are in the scorecard on arcprize.org.

What is still unknown / limitations

The evidence base is narrow: one benchmark (the public slice of ARC-AGI-3), one model (Claude Opus 5), one skill, and no head-to-head comparison with RL or fine-tuning, and no evaluation on other benchmarks, so the generalization that skill engineering has become a competitor to retraining is a single data point, not a pattern. The result is a verified demo on a closed public slice, not a production service: there is no data on cost and latency, and the pattern cannot yet be built into a production budget. The ceiling score was achieved on public games and does not guarantee transfer to other tasks and benchmarks.

Sources

Author

Look at AI, editorial team