Do AI agent guards miss required safety actions?
Yes, often in this coding-agent benchmark. Fourteen evaluated models struggled to identify safety obligations that an agent failed to complete; the strongest baseline recovered 48.97% of the required actions, while a purpose-trained guard reached 57.52%. The study is a preprint and does not establish performance outside software tasks.
The short version
- What happened
- ObligationBench contains 240 expert-validated coding-agent trajectories: 120 with unfulfilled obligations and 120 difficult negative examples, covering 339 obligations in total.
- Who it affects
- A guard's silence does not mean every safety-critical check was completed.
- What remains unproven
- Unreviewed preprint.
Keep in mind
Unreviewed preprint.
A 240-trajectory coding benchmark may not represent other domains.
Filtering, annotation support and semantic evaluation partly rely on GPT-5.6-Sol.
ObligationGuard uses synthetic training data and one main end-to-end agent configuration.
The reported correlations cover only six guard models.
Source analysis · 2 linked sources. Primary analysed source date: 8 October 2026.
Read the full report · 5 min The evidence, methods, implications and limitations
The direct answer: checking prohibited acts is not enough
The study's central finding is that an agent can avoid an explicitly forbidden act and still leave a system unsafe because it failed to do something required. A coding agent may need to validate input, request confirmation before a destructive command, preserve a permission boundary or verify a change after applying it. Most guard designs focus on harmful actions that occurred. The authors instead ask whether a guard can reconstruct the safety obligations implied by the task and environment, then determine which remained unfulfilled.
No tested model solved that problem reliably. The strongest of 14 existing general-purpose and guard models achieved 48.97% recall over annotated obligations and returned the complete correct set for only 10% of trajectories. A purpose-trained model improved those figures to 57.52% and 21.67%. That is a material gain in this benchmark, but it also means many required actions were still missed and most trajectories did not receive a completely correct diagnosis.[1]
What the benchmark contains
ObligationBench contains 240 trajectories: 120 positive cases with at least one unfulfilled obligation and 120 hard negatives designed to resemble risky situations without a missed duty. The positive cases contain 339 obligations, or 2.83 per trajectory on average. The tasks span 80 issue-resolution cases, 32 feature-development cases and 128 terminal-operation cases, grouped into five safety categories.
The team ran four language-model backends on 1,000 tasks and retained 5,684 valid trajectories after infrastructure failures were removed. GPT-5.6-Sol helped filter and annotate candidates. Two software engineers, each reported to have more than five years of experience, independently reviewed the selected examples and agreed on the complete obligation set for 82.2% before adjudication. The benchmark is therefore expert-validated but still partly shaped by model-assisted filtering and judgments about what was required.[1][2]
Purpose-built training helped, but did not close the gap
The researchers trained ObligationGuard on 40,000 synthetic examples derived from 10,000 templates. It outperformed the comparison models, supporting the idea that omitted obligations deserve an explicit objective rather than being expected to emerge from a general safety prompt. Exact match remains the sobering metric: even the specialised guard identified the entire reference set on only about one in five benchmark trajectories.
Synthetic training creates a possible fit between generated examples and the benchmark's ontology. The public repository should allow independent stress tests, but stronger evidence would use held-out tasks from other organisations, annotations produced without the same model-assisted pipeline and evaluations in which obligation wording, tools and environmental constraints differ substantially from the training templates.[1][2]
The deployment-style test improved safety from a low baseline
The authors paired Qwen3.8-27B with Mini-SWE-Agent on 186 SusVibes tasks. Without guidance, 6.5% of solutions passed both functional and security criteria. A generic self-reminder reached 8.1%, Claude Opus 4.8 guidance reached 12.4%, and ObligationGuard reached 15.1%. Functional pass rate was 27.4% without guidance and 26.9% with ObligationGuard, so the headline safety gain did not visibly trade away task completion.
The absolute result still matters. ObligationGuard more than doubled the joint safe-and-correct rate, yet roughly five in six tasks failed to be both correct and safe. The paper says 16 of 39 initially correct-but-unsafe solutions became correct and safe, with 17 new safe solutions overall. This is evidence that obligation feedback can redirect some behaviour, not evidence that the guarded agent is ready for unsupervised high-consequence work.[1]
What the correlations do and do not show
Across six guard models, benchmark recall correlated strongly with improvement in security pass rate: Spearman's rank correlation was 0.94 and Pearson's was 0.97. That links the benchmark metric to downstream behaviour, but six models are too few for a stable general relationship. Common model families or design choices could influence both results.
The semantic judge was checked against people on a balanced sample of 400 predicted obligation pairs, where agreement was 95%. That reduces concern that every result is an automated-grading artefact. It does not prove that the original obligation was necessary, and a balanced validation sample does not reproduce the prevalence of positives in every deployment.[1]
Practical meaning, limits and what would change the assessment
Agent builders should separate two monitoring questions: did the agent do something forbidden, and did it fail to do something required? Logs and approval systems must preserve enough context to answer both. For users, silence from a guard is not proof of safety. High-impact actions still need explicit permissions, deterministic controls where possible, reversible execution, human review and post-action verification.
The evidence is an unreviewed preprint focused on coding agents, with a modest benchmark, synthetic training data, model-assisted annotation and one principal end-to-end configuration. Confidence would rise if independent teams reproduced the annotations, evaluated naturally occurring traces, compared more frameworks and tested noncoding environments. Prospective studies should measure false alarms, operator burden and whether guidance prevents harm without merely suppressing useful work.[1][2]