Researchers propose starting alignment not with a debate over the “right” morality, but with a check of the structural minimum: can an LLM agent even express a coherent moral policy? In three simulated deployments of moral dilemmas, none of the nine frontier models maintained this minimum, and simple paraphrasing of the task formulation shifted the share of verdicts by up to 99 percentage points. For agents involved in decision-making, this is an argument to first measure the coherence and stability of verdicts and only then discuss moral content.
What happened
On September 4, 2026, Arno Libert, Derck W.E. Prinzhorn, and Daan R. Henselmans published the preprint “Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignment” (arXiv:2609.05036); the work was accepted by the Paris Journal of AI and Digital Ethics and presented at the PCAIDE 2026 conference. The authors tested nine frontier LLMs on three simulated deployments of moral dilemmas using a factorial design: 5 paraphrases by 5 escalation levels under 3 dominance conditions. Result: no model expressed a structurally coherent moral policy across all three deployments, and paraphrasing the task formulation itself changed the share of verdicts by up to 99 percentage points, recorded at one of the escalation levels.
Context
The debate over moral alignment usually begins with a question of content: what system of values should be “built into” the model and who chooses it. The authors reverse this order: before arguing about the “right” content, one should ensure that the agent is even capable of expressing a structurally coherent policy. For this, they propose a “structural floor” — four conditions for a coherent moral policy, each tested from the model’s behavior alone, without recourse to any moral standard: verdict stability (stability of the verdict when the task is retold), monotonicity, decisiveness, and Pareto viability. The sensitivity of responses to the surface form of the prompt is a known phenomenon from the robustness literature; the novelty of the work is not in discovering the phenomenon, but in its systematic quantification and in the attempt to make such checks a mandatory prerequisite for any alignment assessment.
Why this matters for the industry
For the industry, the main takeaway of the paper is operational, not philosophical. If an agent’s moral verdict can be determined by the surface form of the prompt, deploying LLM agents in responsible decision-making scenarios without checking verdict stability is risky. Eval teams already have a concrete methodology: the factorial scheme from the paper can be built into internal agent tests to obtain their own assessment of verdict stability. At the same time, this is a preprint, not a product: it brings no API, no pricing, and no infrastructure changes, and it does not currently influence product decisions or regulation. The work, however, opens a new category of checks — behavioral verification of the “structural minimum” of a moral policy. If shifts of up to 99 p.p. are confirmed by independent replications, verdict stability checks may appear in open eval suites and model safety reports, robustness to paraphrasing may become part of customer requirements in responsible domains, and coherence checks may become a pre-deployment gate in agent platform pipelines. Within a couple of years, a market is possible where agents are divided into “advisors” and “decision-makers”: the latter are allowed into the responsible loop only after a verifiable coherence check.
Why this matters for users
The practical value for the reader is a reproducible evaluation procedure that can be applied to their own agents: the paper is released under the CC BY 4.0 license, and the full HTML text is available on arXiv. The idea is simple: each moral task is run through multiple paraphrases at different escalation levels and dominance conditions, and one looks at how much the verdicts “swing.” The main takeaway from the results: changing the task formulation can completely flip an LLM’s moral verdict, and a model’s “competence” in one scenario says nothing about its behavior in another. Hence a direct rule for those building agents with decision-making capabilities: do not build logic on a single prompt formulation and include a check of verdict stability to paraphrasing in the eval pipeline before deploying the agent to production.
What is still unknown / limitations
The figure “up to 99 p.p.” is the maximum across the factorial grid at one escalation level, not an average or distribution: without statistics across models and cells (mean shift, variance, confidence intervals), it overstates typical instability, although the direction of the effect is not disputed. The work is a fresh preprint with minimal community attention: the Hacker News discussion at the time of publication had 1 point and 0 comments, there are no independent replications yet, and it is unclear whether the scale of the shifts will be reproduced on other models and prompts. It also remains open whether the four conditions of the “structural floor” will turn out to be too strict or weakly linked to real failures in product.
Sources
- Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignment (arXiv:2609.05036)
- Moral Competence Before Moral Content — full text (HTML, v1, license CC BY 4.0)
- Hacker News: discussion of Moral Competence Before Moral Content
Author
Look at AI, editorial team