Ant International's research team (ant-intl) has released Code2Skill, a pipeline that converts source code from open repositories into 'skills'—procedural knowledge for coding agents—and has published the DeveloperSkills-Code2Skill (CodeSkillBank) dataset on Hugging Face. The bank contains 1,006,822 accepted skill records from 19,769 GitHub repositories with 500+ stars, and each record has been verified through code reconstruction under the supervision of an LLM judge. Connecting such skills provides a measurable gain across a broad 'model × benchmark' evaluation grid. The 619 MB dataset is already available for download, while the pipeline code itself remains closed—the project is in pre-release status.

image
image
image

What Happened

Code2Skill is a pipeline that extracts procedural knowledge ('skills') directly from the source code of open repositories; in parallel, ant-intl published the DeveloperSkills-Code2Skill dataset, known as CodeSkillBank, on Hugging Face. The skills in the bank are of three types—atomic operations, composite workflows, and recurring patterns; at the retrieval card level, this amounts to 750,748 purpose cards and 3,600 verified patterns. Each record stores application conditions, invariants, anti-goals, failure cases, and provenance, and is verified through source-body-blind reconstruction: the model reconstructs the code based on the skill without seeing the source body, after which a source-aware 'judge' compares the reconstruction with the original—this is an LLM consistency check, not a formal proof of equivalence. The effect was measured on a grid of 9 model configurations (DeepSeek DS4-Flash, Qwen3.5/Qwen3.6 27B, Gemini 2.5 Pro, GPT 5.2) and 8 benchmarks (SWE-bench Verified, AIME 2026, HMMT 2025, GPQA, TerminalBench, and others): the macro-average across 72 'model × benchmark' pairs increased from 42.90 to 47.90, improvements were recorded in 57 of the 72 pairs, and all 9 pairs improved on SWE-bench Verified. The dataset is published on Hugging Face, weighing 619 MB, with cards/edges/pattern_annotations configurations and a join by purpose_id.

Context

Until now, skills for coding agents have been mined primarily from trajectories: approaches like Trace2Skill assume that the target agent has already successfully completed tasks, and only then can something be extracted from its experience. Code2Skill takes knowledge from code—it is available before any interaction with a task and verifiable after abstraction, making it an independent and scalable channel for procedural memory, not a duplicate of existing ones. Essentially, this is the first large-scale skill bank of procedural knowledge built from code rather than agent trajectories, and it is primarily a strong resource contribution rather than a methodological breakthrough: the scale of the verified bank is more important here than the novelty of the mechanism itself. The work fits into the line of skill bank research alongside Trace2Skill and ExpeL, providing these methods with a common reproducible point of comparison.

Why This Matters for the Industry

For the industry, the most significant aspect is measurability: the claimed gain covered most 'model × benchmark' pairs in the evaluation grid, and the dataset provides a common point of comparison and ready-made material for retrieval, planning, and coding RL for skill bank construction methods like Trace2Skill and ExpeL. The product lever lies not in reproducing the pipeline, but in consuming the bank: verified cards can be integrated into the retrieval, planning, and post-review of coding agents without waiting for the code to be released. The release changes the layer economics: 'procedural knowledge in retrieval' is commoditized as a free input resource, while the defensible assets remain the proprietary generation pipeline—Ant International keeps it closed—vertical skill banks for specific stacks, and the way they are used in the product. In the coming months, independent verifications, open re-implementations of the pipeline, and the first products around skill search and domain banks are likely; if the code is opened, companies will be able to build their own skill banks for their codebases, which will change the economics of procedural memory, and if not, the dataset will remain a frozen snapshot. In the two-year perspective, skill banks from code may become a standard layer in agent stacks alongside retrieval and RL, and 'skills from code vs. skills from trajectories' may become the axis of method comparison.

Why This Matters for Users

If you are building agents or retrieval pipelines, the experiment is available without waiting: download CodeSkillBank from Hugging Face, set up retrieval by purpose cards, and A/B compare an agent with and without skills on internal tasks at the SWE-bench Verified level. Cards are convenient to connect as context for the planner and primary generation, and anti-goals and failure cases can be turned into checklists for code post-review—for example, checking that the agent does not fall into known anti-patterns. The cost of such an experiment is minimal because skills are consumed as ready-made data and do not require access to generation. The only limitation is that reproducing the bank on your own repositories is not yet possible: only consumption of the ready-made dataset is available.

What Is Still Unknown / Limitations

The bank's verification is probabilistic: the source-aware 'judge' compares the reconstruction with the original as an LLM consistency check, not a formal proof of equivalence, so the bank's quality is limited by the judge's quality. The gain on mathematical benchmarks AIME 2026 and HMMT 2025 from specifically coding skills is an unexpected signal: without ablations with random or empty skills and with context of comparable volume, the mechanism of the gain is not localized; both transfer and contamination are possible—repositories from SWE-bench Verified are popular projects that likely fall within the 500+ star repository sample. The pipeline code is closed: pre-release status, private repository without a license, so there is no independent verification of the pipeline itself. It is correct to limit confidence to the level of 'the dataset as an artifact is verified,' not 'the claims of gain are proven': ablations, contamination tests, and comparison with Trace2Skill and ExpeL on a common base are ahead. Finally, the legal status of skills derived from others' code is not yet defined, and the cost of LLM judge verification at scale may become a limiting factor for the entire skill bank line.

Sources

Author

Look at AI, editorial team