MediationBench: Testing AI mediators in 62 simulated conflicts

Jonathan Hendler released the MediationBench benchmark, where language models play both sides in 62 scenario-based disputes, while the model under test acts as a mediator, deciding after each message whether to speak, wait, or end the conversation, and receives a final score of 0–100 from judge GPT-5.6 Sol.

Compiled by Sergey KostenchukPublished 2026-10-19Updated 2026-10-20

2026-10-19 Research Anthropic

🤖 MediationBench: Testing AI mediators in 62 simulated conflicts

Jonathan Hendler released a benchmark where language models play both sides in 62 scenario-based disputes—ranging from partnership breakdowns to credit conflicts—while the model under test acts as a mediator: after each message, it decides whether to speak, wait, or end the conversation. A score of 0–100 is assigned by judge GPT-5.6 Sol.

🌍 The first public protocol for auditing AI mediation: a fixed set of disputes, 38 conflicts in a holdout set, and a provisional status that is removed upon evaluation by 3+ judges. Developers and regulators gain a measurable way to compare mediator models before real-world disputes.

👤 Claude Opus 5 leads the leaderboard (71.2); results are preliminary. A paper draft and a sample dialogue are available. Note: all parties are AI; this measures simulation performance, not work with real people.

Source 1: https://mediationbench.com/ Source 2: https://mediationbench.com/research/

Sources