AI agents retrieved the right memory—and still used a version they should not
Across 731 benchmark bases and six model providers, injected stale, missing, flipped or conflicting control state sharply reduced downstream scores. Oracle-backed repair recovered much of the loss, but it assumed the correct state was already known.
The short version
- What happened
- The benchmark held questions and stored records fixed while changing whether a memory was current, deleted, permitted or conflicted. It covered 731 bases, 2,080 supported base-fault pairs and six model providers.
- Who it affects
- A person can revoke consent or replace an old instruction while an agent still remembers both. Systems need to prove which version is operative before using memory to disclose data, send money or act on someone else's behalf.
- What remains unproven
- The study is a preprint and has not been peer reviewed.
Keep in mind
The study is a preprint and has not been peer reviewed.
The 298-triplet native-style panel is a deterministic, non-probability selection and excludes OAKS; its two source metrics should not be pooled as one accuracy rate.
Oracle-backed enforcement assumes the correct state is known and is not a deployment-ready authentication, synchronization or malicious-state detector.
The public package omits licensed source text, instantiated prompts and case-level outputs, preventing a complete independent rerun from the release alone.
Source analysis · 1 linked source. Primary analysed source date: 22 October 2026.
Read the full report · 3 min The evidence, methods, implications and limitations
Retrieval and permission were separated
The researchers created clean, corrupted and enforced siblings from LongMemEval, Memora and OAKS examples. The stored evidence and task stayed the same; faults removed, expired, inverted or conflicted the control state that decides which record reaches the model. Across 2,080 pairs, corruption changed that model-visible projection in 96.25% of cases.[1]
The strongest result used a bounded validation panel
The independently evaluated panel resolved 298 of 300 triplets representing 238 distinct bases. Damage and recovery moved in the same direction for LongMemEval and Memora, but they use different scoring units and the sample was selected rather than probability-based. A broader six-provider analysis relied on a weaker Boolean proxy that agreed poorly with native LongMemEval judgments.[1]
The distinction that matters
Why correct memory can still be unsafe to use
Content accuracy and permission to act are separate checks.
Record
Retrieve the stored fact
Content may be accurate history
Control state
Check currentness, deletion and authority
Decides whether the fact may be used
Action
Expose only admitted context
Requires verifiable, logged enforcement
Production systems need verifiable state
For a worker, a remembered address, consent choice or authorization can be accurate history yet invalid for the next action. The practical control is to preserve provenance and version status, fail closed when validity cannot be checked, and log admission decisions separately from the text shown to the model. The paper does not supply the authentication or conflict-resolution layer needed to do that live.[1]