Workplace AI agents struggled when records conflicted—what should teams verify?
Across 1,613 questions over simulated projects, the strongest tested model reached 68% accuracy with 50 retrieved records and remained weak on cross-source and historical conflicts.
The short version
- What happened
- MMEnterprise-Bench contains 1,613 questions over 7,140 simulated workplace records from 30 projects, including chats, emails, reports, slides and figures.
- Who it affects
- A worker who relies on an agent for a deadline, figure or approval could repeat an obsolete value even when the correct revision exists elsewhere in the workspace.
- What remains unproven
- The unreviewed benchmark uses simulated collaboration, LLM-generated records and automated answer judging; no real enterprise deployment or worker outcome was measured.
Keep in mind
The unreviewed benchmark uses simulated collaboration, LLM-generated records and automated answer judging; no real enterprise deployment or worker outcome was measured.
Source analysis · 1 linked source. Primary analysed source date: 20 October 2026.
Read the full report · 3 min The evidence, methods, implications and limitations
More retrieved evidence helped, but did not solve revision conflicts
A new South Korean benchmark finds that workplace AI agents can answer routine questions yet still falter when a team has corrected a figure, preserved an older document or spread evidence across several images. The researchers built 30 simulated collaboration projects from public papers and reports, retaining earlier versions rather than overwriting them. The resulting workspace contains 7,140 records and 1,613 questions: 1,102 evidence-reasoning tasks, 424 conflict-resolution tasks and 87 questions that should be judged unanswerable.
Ten proprietary and open models answered with either 10 or 50 retrieved records. GPT-5.6 Sol led at 68.0% overall with 50 records, but scored 55.2% on cross-source conflicts and 56.1% on temporal conflicts. Its accuracy fell to 27.5% when a question required several images, compared with 83.3% for one text record. A larger retrieval budget improved recall, but stronger retrieval did not consistently produce a correct answer.[1]
The practical control is version-aware review, not blind retrieval
For workers, the immediate implication is procedural: an agent should expose which record version and timestamp support an answer, surface contradictions and allow a person to inspect the evidence before a deadline, metric or decision is reused. Simply giving the system more documents can also give it more obsolete or conflicting material.
The benchmark remains synthetic. LLMs generated and validated much of the workspace, GPT-5.6 Luna judged model answers, and the authors themselves reviewed a 100-item sample; all 50 automatically accepted items passed three human reviewers, but that sample cannot establish correctness for all 1,613 questions. Real-company trials with authentic permissions, revision systems and independent human adjudication would change the assessment.[1]
The distinction that matters
Why a workplace answer can be current, historical or unsupported
The same claim can appear in several sources and several versions.
Cross-source
Chat and report disagree
Agent must identify the applicable record
Temporal
A value changed over time
Question date determines the answer
Unanswerable
Required evidence is absent
Agent should decline to infer