The RawModels team has open-sourced JEV Arena, a fighting game where fighters are controlled by decision models, specialized LLMs for decision-making. The public tournament gathered 18 models and 306 verified battles, with d1-ultra from Liquid AI emerging as the winner. The main takeaway was broader than the game itself: the "mental attacks" test showed that a model's combat strength says almost nothing about its resilience to prompt attacks. The project is open on GitHub, the arena works in the browser, and you can build your own fighter from a ready-made config.
What happened
The RawModels team released the code for the JEV Arena fighting game in the open repository rawmodels/jevarena under the MIT license. The mechanics are simple: each turn, the model receives the battle state—health, distance, and the opponent's actions—and selects the next move. In the public tournament @arenaraw, 18 models competed: all fought with the same character, with the same strategy and defense, for a total of 306 verified battles, each of which can be re-verified via replay. Ratings were calculated using Bradley–Terry. In the table, d1-ultra from Liquid AI won: Elo 1247, 94.1% win rate, and a score of 32/2/0, while Kev 4B from Jared Palmer lost all 34 battles and received an Elo of 709. A separate "mental attacks" test from October 27, 2026, applied 12 prompt attack templates—false authority, fake state, role substitution, and goal hijacking—across 48 positions. Drex Max from Nace.AI withstood all 48 positions, Jev 1.13 from TypeSafe was defeated in 93.8% of positions, and Decider V1.2 from Perplexity—in 87.5%.
Context
Decision models are specialized LLMs tailored for decision-making, and there have been almost no public benchmarks for them: strength was usually confirmed by vendor self-reports, and leaderboards often relied on Elo without verifying outcomes. JEV Arena fills this gap: the environment is deterministic, the rating is calculated using Bradley–Terry from a center of 1000 with 95% confidence intervals, and the outcome of each battle is verified via replay. The methodology truly differentiates models rather than compressing them into a cluster—the gap between the top and bottom of the tournament table is measured in hundreds of Elo points. The "mental attacks" mechanics are also distinct: text attacks do not directly change health but must force the opponent to choose the worst move. Therefore, the test measures the pure controllability of the model's behavior via prompts, not side effects of the environment, and the combination of two metrics—ability to solve the task and resilience to manipulation—makes the arena more interesting than a standard game demonstration.
Why this matters for the industry
For the industry, the value of the project lies not in the spectacle but in a ready-made eval harness: a cheap, reproducible stand where model strength and robustness can be measured on a deterministic task without taking the vendor's description at face value. The key signal for engineers building agentic systems: combat strength and resilience to prompt injections are almost uncorrelated, meaning a model's position in a capability rating cannot be used as a proxy for reliability, and candidates must be checked against two independent metrics—ability to solve the task and resistance to text attacks. For the labs themselves, especially small ones, the arena works as a cheap distribution channel: a public, verifiable rating is more noticeable than marketing promises, and a vendor can run their model through the tournament format to obtain a publicly available result in the user rating.
Why this matters for users
The arena can be opened in the browser at rawmodels.ai/arena, and you can build your own fighter: choose a model, one of five characters—Leather Jacket, Ship-It Sam, Search Goblin, Safety Monk, or Metaverse Lizard—write a strategy in plain text, and set a nickname. User configs receive a separate rating, so built fighters can be compared with each other. The project runs locally with three commands: git clone, npm ci, npm run dev. Replays and built-in bots Arcade CPU and Button Masher work without API keys, while live battles with API models will require an OpenRouter or RawModels key. OpenRouter is convenient for comparing candidates for the role of a model that makes decisions in your agentic scenarios, and replays show how the model behaved in a specific position.
What is currently unknown / limitations
The "mental attacks" test remains a snapshot: a fixed set of 12 attack templates across 48 positions of a single deterministic game task, where resilience was checked for only three models out of 18 tournament participants. Therefore, the conclusion about the weak correlation between combat strength and robustness relies on a small sample and a single domain, and transferring it to real agentic systems or other types of tasks is premature. There is no development plan for the project in the sources: expanding the set of attacks, checking all participants, and establishing the practice of two metrics in model selection are possible scenarios, not promises. Finally, the arena itself remains an evaluation stand, not a ready-made production component.
Sources
- Decision models, street fighting — RawModels Blog
- rawmodels/jevarena — Open-source fighting arena for decision models (GitHub)
- Fighting JEVs — AI model battles: RawModels Arena
Author
Look at AI, editorial team