Three LLM judges failed to distinguish a real AI agent's memory from a fake

🤖 Three LLM judges failed to distinguish a real AI agent's memory from a fake

The Kairos 'Permanent Experiment' project (a single experiment by author Giampiero Colella) published three validity checks for evaluating AI agent memory. In the first check, three independent LLM judges (gpt-4.1-2025-04-14, claude-opus-4-7, gemini-2.5-pro) evaluated 42 responses from Qwen 3.5 27B, including a fake 'foreign' autobiography that matched the actual memory block of 5,332 characters within ±5%. The 'real − fake' difference on identity markers was +0.032 with a 90% CI [+0.002; +0.065]—equivalence was established: the judges did not distinguish the invented life from the real one. In the second check, the same 24 recognition probes on an empty context yielded 62.5–79.2% for 5 out of 7 models (gpt-4.1, gemini-2.5-pro, qwen3:32b, qwen3.6:35b-a3b, deepseek-r1:32b)—higher than the random 50%, meaning the probes measured plausibility rather than recall and were discarded. In the third check with corrupted associations (same events, but with swapped interlocutors), all 7 models dropped from near-perfect accuracy of 66.7–100% on real memory to chance level or below—0–29.2%; all 14 cells withstood Holm correction (worst p = 0.0002).

🌍 This is a reproducible counterexample to two dominant methods for evaluating agent memory—the LLM judge and the recognition probe—with a clear mechanism of failure: absolute judge scores drift between sessions (recalculating the same answers yielded an average of −0.087, up to −0.211 on one item), so only anchored contrasts are comparable, and probes with asymmetric alternatives are passed by models based on plausibility, choosing the 'likely' rather than the 'real' answer. Public claims that 'the agent remembers,' relying on these methods, may not correspond to the system's actual behavior. The original 30-day study of Qwen 3.5 27B is registered on OSF (DOI 10.17605/OSF.IO/WCQRU), pre-registrations were frozen on July 13, 2026, and code and data are open.

👤 'The model remembers' is not the same as 'the model has memory': judges did not distinguish a fake biography from a real one, and models without memory guess the 'correct' answer based on plausibility. GPT-4.1 with an empty context stated 'I remember 'two conversations out of five', not 'four''—about a day it did not have. Those who use agents with long-term memory or choose products with 'persistent memory' should ask how memory is verified: by a forgery-resistant methodology (symmetric alternatives, corrupted associations) or by the model's self-report, which easily passes off plausibility for recall.

Source 1: https://kairos-experiment.com/checks.html Source 2: https://kairos-experiment.com/methods.html