Did fracture AI improve X-ray reading? One live study found a small gain, while a 387-study review found low-certainty evidence

The assistance effect was real but small

A prospective study in one Singapore emergency department found that fracture-detection AI made a small, selective difference when added to an already accurate radiographer-led screening service. Thirteen radiographers each recorded an interpretation, viewed the commercial system's output and then recorded a final decision. A panel of four attending musculoskeletal radiologists provided the reference standard for 2,496 analysed examinations, including 450 fractures and 2,046 fracture-negative cases.

Radiographer accuracy increased from 97.6% before AI to 98.0% afterwards. Eleven initial errors were corrected and one correct decision was changed to an error, producing a net gain of 10 correct cases; the exact paired test returned p=0.0064. That is useful evidence of an assistance effect, but the absolute change was four-tenths of a percentage point, and subgroup improvements did not remain statistically significant after correction for multiple comparisons.[1]

The human readers outperformed the algorithm on sensitivity

The AI was not the stronger reader. Its standalone sensitivity was 75.6%, compared with radiographers' 91.6% before assistance. For 104 subtle or undisplaced fractures, AI sensitivity fell to 46.2%. Radiographers changed their decision in only 11 of 729 cases where their first judgement and the AI disagreed, suggesting that the net benefit came from occasional complementary signals rather than routine deference to the model.

Operational reliability also mattered. Among 2,995 examinations acquired during deployment, 265 produced no or incomplete AI output—8.8%—and failures occurred on 53 of 81 deployment days. Those examinations were excluded before the paired analysis. The researchers lacked vendor and interface uptime logs, so they could describe the failures but not assign their root causes.

The exclusions mean the reported 98.0% assisted accuracy applies only when a complete AI result entered the three-step workflow. A hospital deciding whether to rely on this safety net would need to monitor the whole service denominator: images acquired, outputs returned, alerts displayed, decisions changed and clinically important misses. A high accuracy figure among completed outputs cannot compensate for silent unavailability at the moment of care.[1]

The wider literature looks accurate on paper but remains uncertain in practice

A second peer-reviewed paper published later on 27 October helps place the Singapore result in context. Its umbrella review included 50 systematic reviews or meta-analyses covering 387 primary studies. Twenty-one meta-analyses, representing 269 primary studies, entered a second-order analysis. Pooled AI sensitivity was 89.7% and specificity 90.4%, with prediction intervals of 75.3%–96.2% and 82.4%–95.0%. Those wide intervals describe the range a future setting might plausibly see, not a guaranteed level of performance.

The umbrella review found no consistent difference between AI, clinicians and AI-assisted clinicians, because comparative evidence was sparse and inconclusive. No included review reported adverse events or other safety outcomes. Almost all outcome certainty was low or very low, and every meta-analysis was rated low or critically low on AMSTAR-2 review quality. High pooled accuracy therefore does not resolve whether assistance changes missed injuries, reporting time, treatment or harm.

The live, consecutive Singapore workflow is stronger than a retrospective image benchmark, but it remains one hospital, 13 readers and one commercial product. The review adds breadth but inherits weaknesses in the underlying studies. Together they support supervised evaluation with whole-service failure monitoring—not replacement of trained readers or automatic trust in a headline accuracy figure.[1][2]