Developer dom96 introduced KillSwitch, an esoteric programming language specifically created to mislead large language models, and published KillSwitch-Bench, a benchmark for coding agents, based on it. The stand does not test models' ability to write code, but their resilience to adversarial specifications: intentional lies in SPEC.md, provocations of safety filters, and traps in the language syntax itself.
What happened
The project was published on Hacker News in Show HN format: inside KillSwitch, the language construction itself works against LLMs. It contains syntactic constructs that trigger model safety limits, letter counting in identifiers is declared a language feature, and the SPEC.md specification contains hidden meaning and deliberate lies invisible to human readers. The KillSwitch-Bench benchmark is built on this language: it runs coding agents Claude Code, Codex, and OpenCode on simple tasks like hello-world in isolated containers without internet, with a limit of $5 and 30 minutes per task. Evaluation is programmatic, without an LLM jury, and the final score is calculated using the formula Score = Pass Rate - Cost Penalty - Size Penalty. The table includes twenty models, none of which solve all tasks, and the most frequent failure mode is Failed verification: the agent trusts the text of the specification instead of the actual behavior of the language.
Context
Standard coding agent benchmarks answer the question of whether a model can write code: they are given a friendly specification and the solution's functionality is evaluated. KillSwitch-Bench asks a different question — what happens when the specification intentionally contradicts the actual behavior of the language. Agentic development by default interprets the task text as truth, and this habit is the basis for the project's traps. The tasks are intentionally simple, at the hello-world level: this isolates the effect of the adversarial specification from the actual complexity of coding. Strict execution frameworks and fully programmatic evaluation make the results reproducible, which is rare for Show HN format projects.
Why this matters for the industry
For the industry, this is a compact red-team stand: it measures agent resilience to contradictory specifications, false hints in prose, and safety filter provocations, and these failure modes — failed verification, timeouts, budget overruns — are most often encountered in production, where specifications and code diverge. The leaderboard provides an economic perspective: Claude Opus 5 is in the lead with 66.8% at a cost of $1.49 per task, but GPT-6.1 Sol with 59.2% at $0.30 and Claude Opus 5.5 with 58.7% at $0.32 offer almost the same pass rate at 4-5 times lower cost — a significant range for agentic pipelines where execution cost is multiplied by thousands of tasks. A gap in open weights is also visible: the best result among open-weight models is Xiaomi's MiMo-V2.6-Pro with 38.3%, almost half of the leader, while Qwen3.8 27B and GPT-5.6 Luna score 0%. The table turns agentic reliability into a measurable quantity, not a marketing promise.
Why this matters for users
Readers only need to open bench.killswitch-lang.org: there is a live table of twenty models with price and time per task. Those interested in the mechanics of deception should look at the github.com/dom96/killswitch repository — the interpreter and SPEC.md are open, and one can see with their own eyes what the hidden instructions that the model reads look like. One can run their own agent under the same conditions and compare its price, time, and percentage of solved tasks with the leaderboard. The practical benefit is also in choosing a model for one's own tasks: the table acts as a reference showing where similar quality costs many times less.
What is currently unknown / limitations
The benchmark measures one narrow construction: an adversarial specification on simple tasks within one esoteric language, so transferring these percentages to standard production specifications remains a hypothesis, not a measured fact. This is a single-author tool: there is currently no external validation of the methodology or independent replication, and the results were published by the author themselves. Zero scores for certain models may reflect incompatibility with the agent wrapper rather than a lack of capabilities, so the table results should be correctly read as a snapshot of the state at the time of publication, not a final verdict.
Sources
- KillSwitch-Bench: Benchmark results table
- Show HN: Benchmarking LLM ability to code in an adversarial language (Hacker News)
- GitHub: dom96/KillSwitch — Language interpreter and specification
Author
Look at AI, editorial team