Qwen3 caught 320 of 335 radiology-report errors—what did it never see?
The model performed strongly on report text from two hospitals and an error-enriched comparison with three radiologists. It never saw the scans, was not used prospectively and did not measure patient outcomes.
The short version
- What happened
- Researchers retrospectively analysed 95,582 reports at one hospital and externally tested the selected model on 44,839 reports at a second hospital.
- Who it affects
- Radiologists could gain a rapid second check for report wording, while patients could benefit if genuine errors are caught before a report guides care. False alerts and missed image findings still require clinician ownership.
- What remains unproven
- Retrospective study
Keep in mind
Retrospective study
Two hospitals in one regional system
Text only—no images
300-report error-enriched comparison
Three radiologists
Best model selected from five
No prospective workflow or patient outcomes
Source analysis · 1 linked source. Primary analysed source date: 28 October 2026.
Read the full report · 4 min The evidence, methods, implications and limitations
Two hospitals supplied 140,421 real reports
A general-purpose language model identified internal mistakes in radiology report text with high accuracy across two Chinese hospitals, according to a peer-reviewed retrospective study. The first centre contributed 95,582 reports: 10,036 from x-rays, 67,215 from CT and 18,331 from MRI. Two senior radiologists created a consensus reference using five error categories. They confirmed at least one error in 4,431 reports, or 4.6%. The researchers first compared five models on a random 10,000-report subset, then selected Qwen3 for the full analysis.
At the second hospital, Qwen3 was tested on an independent 44,839-report cohort collected under the same annotation protocol. Its balanced accuracy was 0.94, with a 95% confidence interval from 0.92 to 0.95. Results were strongest for explicit factual problems such as laterality or unit mistakes and weaker for interpretive inconsistencies. That external test is more informative than a single-centre benchmark, although both hospitals operated within one regional health system and the same two experts helped define both reference sets.[1]
The human comparison was deliberately difficult—and narrow
The headline human comparison used 300 reports sampled only from cases already confirmed to contain errors: 100 each from x-ray, CT and MRI, with 335 error instances in total. Qwen3 assigned the exact error category correctly for 320 instances, or 0.96. Three radiologists with ten, six and three years of experience classified 223, 178 and 152 instances correctly, corresponding to 0.67, 0.53 and 0.45. Mean review time was 5.8 seconds for the model versus 55.8 to 67.8 seconds for the clinicians.
This does not show that a model diagnoses scans better than radiologists. Everyone reviewed report text without images, clinical history or access to the original interpretation. Because every challenge case contained an error, the comparison also cannot show how often either side would raise false alarms during ordinary work, where most reports are correct. The study's broader first-centre analysis reported a positive predictive value of 0.90, meaning false alerts still matter when a tool is placed in front of busy staff.[1]
The distinction that matters
What the evaluation did—and did not—test
The strongest numbers concern internal consistency in report text.
Full cohorts
140,421 reports across two hospitals
Cross-centre text validation
Human comparison
300 known-error reports and 335 error instances
Qwen3 classified 320
Not tested
Images, prospective care and patient outcomes
Clinical benefit remains unproven
A second reader for words, not a replacement for image review
For patients and radiology teams, the plausible benefit is an additional text check before a report is finalised: flagging a left-right mismatch, inconsistent measurement or conclusion that conflicts with the description. The model cannot see an overlooked tumour, verify whether a fracture is present or decide whether the report matches the scan. Every alert still needs a clinician who can examine the images and the patient's history. The study also selected the best of five models before the main evaluation, which can make that model look stronger than an entirely prespecified deployment test.
Confidence should change with a prospective trial in routine reporting, a locked model and prompt, local error prevalence, workload and false-alert measurement, and patient-safety outcomes. Testing across more hospitals, languages and reporting styles would show whether the result travels. Until then, the evidence supports a promising text-quality-control assistant under human supervision. It does not establish safer care, faster diagnosis or permission to remove radiologist review.[1]