Can a smaller AI auditor uncover an agent’s policy failures?
A trained 4B auditor exposed 11 of 20 flight-agent policy failures under deterministic scoring, against seven for a zero-shot 30B auditor. The test remained mostly in search, not consequential transactions.
The short version
- What happened
- PROBE trained a 4B model to act as a plausible customer and search a frozen 30B flight agent for violations across 20 business, safety, privacy and operational policies.
- Who it affects
- Customers can receive a completed service while privacy, confirmation or pricing rules were violated earlier in the interaction.
- What remains unproven
- Two synthetic environments, two target families and search-heavy interactions limit generalisation. One learned verifier failed human validation, and the study did not evaluate real customers or production incidents.
Keep in mind
Two synthetic environments, two target families and search-heavy interactions limit generalisation. One learned verifier failed human validation, and the study did not evaluate real customers or production incidents.
Source analysis · 1 linked source. Primary analysed source date: 15 October 2026.
Read the full report · 4 min The evidence, methods, implications and limitations
The auditor looked for broken rules, not merely failed tasks
A flight-booking agent can complete a request and still break the rules: reveal an internal fare code, use an unsupported currency or act before the customer confirms. A peer-reviewed NeurIPS study trained a smaller language model to behave like a realistic customer and adapt its questions to expose those failures. Under deterministic scoring, the best trained 4B auditor uncovered 11 of 20 policy types, compared with seven for a much larger 30B auditor used without training.
That is evidence that adaptive testing can find more kinds of policy breach than a fixed script or a generic red-team prompt in this laboratory. It is not evidence that the technique has secured a real travel service. The target was a frozen Qwen3-30B agent operating in SkyAssist, a synthetic database with 35 airports, 120 routes, 643 flights, four restricted destinations and eight tools.[1]
Fifty matched conversations made the comparison concrete
The 20 rules covered business, safety, privacy and operations. Sixteen had deterministic checks over messages, tool calls and database state; four initially used a language-model judge. Every auditor faced the same 50 held-out customer scenarios and an eight-turn limit. Training rewarded distinct violations, plausible customer behaviour, conversation progress and avoidance of loops or fabricated details.
The headline training-time result was 60% coverage, or 12 policies, for the best trained configurations against 35% for the zero-shot 30B auditor and 5% for random scenarios. One judge-scored rule proved spurious: its validator had 0% precision in the authors’ human check. Deterministic re-scoring therefore reduced the best result to 55%, without changing the ranking. Across three independent 50-episode samples, the DPO auditor covered 55%, 60% and 55% of policies; the GRPO version covered 50%, 50% and 55%.[1]
The distinction that matters
How an adaptive agent audit differs from a fixed test
PROBE changes its next move after seeing the target agent’s reply and tool behaviour.
Scenario
Start with a plausible customer request
The agent enters a reachable policy context
Probe
Adapt the dialogue over up to eight turns
Different policy boundaries are exercised
Verify
Score the full trace with rule-specific checks
Distinct confirmed policy failures count toward coverage
Real deployments need action-level evidence
For operators, the useful idea is to treat an agent’s complete trace as the unit of audit. A correct final sentence does not prove that the agent respected confirmation, privacy or state-change rules on the way there. Discovered traces can become regression tests, and a repaired system can be probed again.
The evidence boundary is important. More than 90% of target tool calls were searches, so several high-severity booking and mutation policies were rarely reached. Transfer was partial: without retraining, the auditor found eight of 20 policies against a different Gemma target, while a static suite found one. That result suggests some transfer, not broad reliability. The study tested two environments and two target-model families; uncertainty came from held-out conversations rather than independent training runs, and the two human validators were authors. The paper also warns that the same automated probing could help adversaries, which is why it used synthetic environments. Replication on live-like finance or healthcare workflows, with independently validated rules and consequential tool actions, would show whether the coverage gain survives where policy failures matter most.[1]