Can clinicians see the evidence behind approved diagnostic AI?
A peer-reviewed audit found public, device-specific performance evidence for 30 of 77 approved pathology and haematology AI products. The result is a transparency finding—not proof that the other 47 lack regulatory evidence or that the documented products are clinically equivalent.
The short version
- What happened
- The authors identified 77 regulated devices from 29 manufacturers: 68 for digital pathology and nine for haematology morphology. All had European market access, while eight also had US authorization.
- Who it affects
- Pathologists and laboratories need visible evidence to judge whether software tested elsewhere fits their specimens, scanners, patients and workflow.
- What remains unproven
- The analysis measured publicly identifiable evidence and did not access confidential regulatory or manufacturer files; the 47-device gap cannot be interpreted as evidence of no validation.
Keep in mind
The analysis measured publicly identifiable evidence and did not access confidential regulatory or manufacturer files; the 47-device gap cannot be interpreted as evidence of no validation.
The landscape cutoff was 31 March 2026, six months before publication, so later certifications, papers or product changes are outside the denominator.
The dual-authorized group contained only eight devices, and observed jurisdiction differences may reflect product maturity, company resources or disclosure practices.
Most documented evaluations were retrospective, with heterogeneous units, comparators and metrics that prevented pooled performance comparison.
One author performed device identification, extraction, verification and analysis, and incomplete European databases make omissions possible.
Source analysis · 3 linked sources. Primary analysed source date: 5 October 2026. Earliest linked supporting source: 5 May 2017.
Read the full report · 8 min The evidence, methods, implications and limitations
The audit measured public visibility, not regulatory approval quality
The researchers asked a practical question that falls between regulation and clinical procurement: when a laboratory, pathologist or health-technology assessor considers an approved AI product, can they find quantitative, device-specific evidence in public? They assembled a landscape of diagnostic software for digital pathology and haematology morphology using the FDA's AI-enabled device list, a previous open device register, manufacturer portfolios, scholarly searches, conference proceedings and regulatory sources. The cutoff was 31 March 2026. After excluding unauthorized, research-only, hardware-embedded and radiology products, the final denominator was 77 devices made by 29 companies.
Every included product had European market access through IVDR certification or a legacy IVDD route; 45 were IVDR-certified and 32 held legacy certification. Eight also had US authorization—six through 510(k) and two through De Novo—leaving 69 products approved only in the European sample. Approval still implies that evidence was submitted and assessed under the relevant process. The study did not inspect confidential technical files or notified-body dossiers, and it did not retest the products. Its finding is about what an outside decision-maker can independently locate, interpret and compare.[1][2][3]
Thirty of 77 devices had identifiable public performance evidence
The search found public, quantitative, device-specific performance evidence for 30 of the 77 devices, or 39%. For 47 devices, or 61%, the researchers did not identify such evidence. Marketing brochures and qualitative claims without device-level results were excluded. The search hierarchy favoured peer-reviewed articles, then conference abstracts, preprints and regulatory documents. Of the 30 devices with evidence, 25 had a peer-reviewed article as their highest-level source, three were supported by a conference abstract, one by a preprint and one by a regulatory filing.
That denominator needs careful language. “No identified public evidence” is not the same as “no evidence exists,” “the device is unsafe” or “the authorization was unsupported.” Data may sit in non-public regulatory submissions, manufacturer files or sources missed by the search. Conversely, the presence of a paper does not prove that a device will work in every laboratory. Public evidence matters because hospitals must judge whether a study matches their patients, tissue preparation, staining, scanner, workflow and clinical threshold—and because independent scrutiny is difficult when results cannot be seen.[1]
US-authorized products were more visible, but the groups were small
All eight devices authorized in both Europe and the United States had public evidence, compared with 22 of 69 devices in the Europe-only group, or 32%. Fisher's exact test returned p below .001. Haematology morphology products were also more likely to have public evidence—seven of nine, compared with 23 of 68 digital-pathology products—with p equal to .02. Those contrasts describe the audited sample; they should not be treated as causal proof that one regulator or specialty produces better technology.
The eight-product transatlantic group is small and may differ in company resources, product maturity, intended use or publication strategy. Regulatory pathways also create different public documents. The comparison between legacy IVDD and newer IVDR certification was not statistically significant: 14 of 32 legacy devices and 16 of 45 IVDR devices had public evidence, with p equal to .49. Publication lag may matter for recently certified products. The authors therefore resist using the result as a verdict on IVDR and argue instead for structured, consistent disclosure across pathways.[1]
The available studies were mostly retrospective and hard to compare
Among the 30 evidence-supported products, 25 evaluations were retrospective. Sample size could be extracted for 28: the median was 211 patients, slides or specimens, with an interquartile range from 103 to 720 and a range from 30 to 13,349. Fifteen of those 28 evaluations used fewer than 300 units. Those counts cannot be pooled naively because a patient, whole-slide image and specimen are different denominators, while disease prevalence and sampling strategy affect apparent performance. Few studies were prospective or multicentre, leaving selection bias and optimistic local validation as continuing concerns.
The audit found 15 primary metric types grouped into discrimination, agreement, correlation and prognostic performance. Seventeen devices used a discrimination metric, seven agreement, three correlation and three prognostic outcomes. Sensitivity appeared for ten devices, but six of those did not report a paired specificity; no device reported specificity without sensitivity. That asymmetry can make clinical trade-offs difficult to see. A high sensitivity may reduce missed cases while increasing false positives, workload or unnecessary review. Without paired metrics, thresholds and confidence intervals, a procurement team cannot reconstruct that balance.[1]
Assisted-reader studies suggest promise, not a product ranking
Most documented products were compared with a pathologist, expert or manual reference. Eight studies compared AI-assisted with unassisted readers, but only five supplied paired accuracy results. Those studies reported improvements on at least one outcome for products including Paige Prostate Detect, Lunit SCOPE PD-L1, Paige Lymph Node, Ibex Galen IHC HER2 and Deep Bio DeepDx Prostate CNB. Some differences were statistically significant and one study reported shorter review time. Yet endpoints, readers, specimens and designs differed, and several results were small or not statistically significant.
None of this creates a head-to-head league table. Only seven devices shared the same broad combination of agreement metric, retrospective single-centre design and AI-versus-pathologist comparator; eleven had unique design-and-metric combinations. A percentage from one product cannot safely be placed beside another when disease mix, reference standard, scanner, reader experience and operating threshold differ. The useful signal is narrower: some products have credible evidence that assistance can improve a defined task, but the public record is not standardized enough to compare the market or estimate system-wide benefit.[1]
Funding, conflicts and method limits shape the conclusion
The work was funded by the EU Horizon Europe EU4MEDTECH project; the authors state that the funder had no role in the study. Two authors disclosed share ownership in Cellbytes Ltd, and one also reported research funding or consultancy relationships with several pharmaceutical companies; the remaining authors reported no competing interests. Those disclosures do not invalidate the audit, but they should accompany interpretation because the paper concerns commercial diagnostic products and access to market evidence.
The authors describe a systematic landscape analysis rather than a registered systematic review. One author conducted device identification, extraction, verification and analysis, which increases the value of independent replication. European device discovery is also difficult because EUDAMED participation remains incomplete, and search results depend on manufacturers' naming and publication practices. The team used two consecutive search rounds that yielded no new devices as a completeness check, but cannot prove that every eligible product or obscure evidence source was captured.[1]
What would change the assessment
Confidence would rise if regulators or manufacturers published a stable, device-level evidence summary for every product: intended use, population, specimen and scanner, study design, sample denominator, comparator, thresholds, sensitivity and specificity with uncertainty, external validation and post-market updates. IVDR's summaries of safety and performance for higher-risk devices can help, but coverage and accessibility need to become consistent. The EU AI Act may add transparency and governance duties, yet it does not replace device-specific clinical evidence under IVDR or the Medical Device Regulation.
For individual products, the decisive next evidence is prospective, multicentre and independent validation in routine workflows. Studies should report reader assistance against unassisted practice, time and correction burden, false-positive and false-negative consequences, performance across laboratories and patient groups, failures after software updates, and post-market safety signals. Until that record improves, approval should be read as permission to market within a defined regulatory pathway—not as a guarantee that clinicians can readily inspect comparable public evidence or that products are interchangeable.[1][3]