On September 3, 2026, a paper titled "Compile by Training" by researchers from the University of Waterloo was published on arXiv, presenting a neurocompiler that extends the Program-as-Weights (PAW) approach: instead of calling an expensive large model for every input, a text description of a task is converted into a small local artifact in about a minute, which is executed by a local interpreter without teacher calls.

image
image

What happened

The paper, published under number 2609.04199, is by Yuntian Deng, Pengyu Nie, and Stuart Shieber. In the working method, a teacher model (e.g., gpt-5.5) generates training examples from a text description of a task, after which a general local interpreter of about 0.6B parameters is fine-tuned on them via LoRA in about a minute. The result is a .paw file containing a lightweight adapter, which the interpreter executes locally without calls to the teacher model. On the FuzzyBench-Hard subset, where the fast PAW compiler did not give a single exact match, the method achieves 83.6% semantic accuracy. The compiler code is open on GitHub under the MIT license: to work, Python 3.10+, uv, and an OpenAI API key are required, and the default recipe is designed for an accelerator with about 40 GB of memory. A public demo playground is available at programasweights.com/playground?compiler=paw-ft-bs48.

Context

The original Program-as-Weights project, which the new method extends, already has a fast compiler: it "compiles" a task in a single forward pass, i.e., in seconds and without additional training. Compile by Training replaces this instant mechanism with training: the teacher model forms a set of examples, and the interpreter distills the task into a small LoRA adapter, i.e., into a permanent artifact, not a one-time calculation. The approach is applicable to tasks that can be formalized as a "input → output" function: classification, parsing, PII masking, translation. Among the demonstrations from the paper are a translator into the Claudish language, a 3D avatar built from a natural description, and a local PII masker.

Why this matters for the industry

For the industry, this is a ready-made working pattern for replacing expensive remote LLM calls for every input with a local artifact: a small adapter on top of a general interpreter, which can be stored, versioned, and combined like a regular file. This is an alternative to fine-tuning entire models for narrow text tasks, and the market signal is that the layer of narrow text functions is becoming cheaper and faster. Since the compiler itself is open under the MIT license, it is impossible to protect the mechanism, and differential value will focus on specific .paw artifacts and pipelines built around them.

Why this matters for users

Right now, you can open the public playground or clone the MIT repository github.com/programasweights/compile-by-training, build a .paw for your own task, and compare the result with the fast PAW compiler using the benchmark from the paper. This is specifically useful for those who solved repetitive text tasks — classification, PII masking, parsing — with static rules or remote API calls. The practical barrier to entry is hardware for the default recipe and an OpenAI API key for the teacher; for teams without such an accelerator, the playground and benchmarks from the preprint remain for now.

What is still unknown / limitations

The figure of 83.6% is claimed on the FuzzyBench-Hard subset, where the basic fast PAW compiler did not give a single exact match, so the result needs independent verification on other datasets and with other teacher models. Live demo services are deployment examples, not controlled benchmarks. The available materials do not contain data on the latency and reliability of .paw artifacts in production use, there is no information about a commercial product or pricing model, and the teacher stage is tied to the OpenAI API.

Sources

Author

Look at AI, editorial team