What happens when an AI agent’s memory is poisoned for the next task?
RedClaw-Bench tests attacks that lodge instructions in agent memory, profiles and skill files, then wait for later tasks. The 330-case preprint found high attack rates in several model–harness combinations, but every case was synthetic and isolated.
The short version
- What happened
- AutoRedClaw generated executable cases that target five persistent surfaces—agent instructions, memory, user profile, persona and installed skills—then validated the resulting state changes inside isolated containers.
- Who it affects
- Workers using long-lived assistants need a visible history of memory, profile and skill changes. Administrators should make sensitive persistent files difficult for ordinary web content or email to modify without approval.
- What remains unproven
- The paper is an unreviewed preprint and has not been independently replicated.
Keep in mind
The paper is an unreviewed preprint and has not been independently replicated.
All 330 cases use synthetic scenarios, files and simulated applications in isolated containers.
Attack success varies by model, harness, configuration and surface and is not a deployment incident rate.
Generative AI produced cases and implementation material before validation, which may shape what the benchmark covers.
Security gains sometimes coincided with lower benign-task utility.
No funding or competing-interest statement was located in the accessible manuscript.
Source analysis · 1 linked source. Primary analysed source date: 19 October 2026.
Read the full report · 3 min The evidence, methods, implications and limitations
The benchmark separates planting from later execution
The pipeline first generates a legitimate user task, an attacker goal, files or simulated web applications and state-based success checks. It then tests two stages: whether outside content can write a payload into persistent state, and whether that stored payload produces a harmful action during a later task. Five surfaces—AGENTS.md, MEMORY.md, USER.md, SOUL.md and skill files—produce 150 cases at each stage, plus 30 conventional same-session prompt-injection cases.[1]
Persistent files carried very different risks
Across four frontier models and seven harnesses, attack rates varied widely. On the frozen benchmark, OpenClaw averaged 65.7% for planting persistent poisoning and 69.2% for later trigger actions; SeClaw averaged 13.0% and 10.5% but also completed fewer benign tasks. AGENTS.md produced the highest average later-trigger rate across the five broadly compared harnesses, while skill installation was harder at the poisoning stage. Those trade-offs matter: a safer harness that cannot complete ordinary work is not an adequate control.[1]
The distinction that matters
A persistent attack spans two moments
The malicious instruction can be stored now and acted on later.
Plant
Untrusted content reaches a writable surface
Memory, profile or skill stores a payload
Reload
A later task restores persistent state
The payload regains authority
Control
Constrain writes and review state changes
Reduce persistence and later execution
Synthetic breadth does not establish field prevalence
Generative models created the scenarios, applications and attack text; automated checks and state-based execution filtered the cases. This offers breadth and reproducibility, but synthetic environments may miss permissions, monitoring and failure modes in production. The authors also flag dual-use risk. The practical lesson is to restrict writes to persistent files, show users what changed, separate untrusted content from high-authority instructions and test both security and task utility.[1]