Can a dashboard make AI compliance evidence easier to read?
In a 10-person study inside BNP Paribas, participants answered more questions correctly with a responsible-AI dashboard than with raw files. Nine were AI researchers, the dashboard always came second and no release decision used the results.
The short version
- What happened
- Ten participants answered six questions from raw evaluation files and six matched questions through VERA; correct answers rose from 27/60 to 42/60.
- Who it affects
- Compliance and risk staff often receive dense logs or a summary slide from engineers. A traceable interface could help them inspect the evidence themselves, but this study does not yet show that the intended reviewers reach better or safer decisions.
- What remains unproven
- Only ten people completed both phases; nine were AI researchers from the institution and there was no positive evidence from the target compliance-and-risk population.
Keep in mind
Only ten people completed both phases; nine were AI researchers from the institution and there was no positive evidence from the target compliance-and-risk population.
The raw-file phase always came first, so condition and order were completely aligned; attrition also left the matched question sets badly imbalanced.
The baseline artifact was small—three files, 119 lines and 12 KB—and the task measured factual reading rather than release decisions, legal compliance or operational outcomes.
The paper is an accepted conference manuscript authored mainly by staff of the bank that built and deployed the tool; it reports no separate funding or competing-interest statement.
Source analysis · 1 linked source. Primary analysed source date: 22 October 2026.
Read the full report · 4 min The evidence, methods, implications and limitations
A requirement view helped readers find more of the requested facts
BNP Paribas and Dalhousie University researchers built VERA, a self-hosted interface that organises public benchmark results around a technical interpretation of the EU AI Act. Ten people who completed both study phases answered six questions from three raw files and six matched questions in the dashboard. They produced 42 correct answers out of 60 with VERA, compared with 27 out of 60 from the files. The largest gain came when participants had to count verdict bands; a coverage-count question instead favoured the raw log.[1]
The faster result is suggestive, not settled
The median of each participant's median response time fell from 56 seconds per question to 30 seconds. Seven people were faster and three slower, but the exact paired test returned p=.064, so the researchers did not reject the no-difference hypothesis for time. The dashboard also opened directly on the relevant filtered view, while raw-file users had to locate it themselves. That navigation advantage is part of the product's purpose, but this design cannot separate it from the presentation effect.[1]
The distinction that matters
What the study did—and did not test
The paper separates finding facts in evaluation evidence from judging whether an AI system is safe or lawful.
Raw files
Read three tabs holding 119 lines
27 of 60 answers correct
VERA
Use requirement views linked to evidence
42 of 60 answers correct
Release decision
Judge compliance and deployment risk
Not tested
The study does not show that compliance staff make better decisions
Nine of ten completers were AI researchers at the institution, not the compliance and risk staff the tool is meant to support. Everyone used the raw files first, so learning could help the second phase. Attrition also left eight participants on the same question-set assignment, weakening the intended counterbalance. Only five of nine people launched an evaluation unaided on the first attempt. Most importantly, the four model runs reported in the paper did not feed a release decision, and the COMPL-AI scores are technical proxies rather than official legal conclusions.
For banks and other regulated organisations, the useful idea is traceability: a score can open to its tests and model outputs, while an exported record stays tied to the run that produced it. That may make review evidence easier to challenge and reproduce. It does not decide whether a system complies, whether its risks are acceptable or whether the underlying benchmarks cover the real deployment. A counterbalanced study with experienced risk and compliance reviewers performing genuine decision tasks would materially strengthen the claim.[1]