AI caught every poor radiotherapy contour in a 27-case test—but flagged six acceptable cases too
A multi-centre prostate-cancer trial tested whether model-to-clinician contour differences could focus expert quality checks. The hold-out result is promising, but it rests on few deviations and did not prove a safer or faster clinical workflow.
The short version
- What happened
- The retrospective study included 107 treatment plans from seven centres: 89 physician contours were rated per protocol and 18 had a minor deviation or were unacceptable.
- Who it affects
- For trial teams, a reliable triage layer could concentrate scarce expert time on the plans most likely to breach a protocol. For patients, the relevant outcome is not a benchmark score but whether the system helps catch consequential planning errors without allowing others through.
- What remains unproven
- Retrospective analysis
Keep in mind
Retrospective analysis
107 usable cases from one phase II trial
Only 18 deviations overall
27-case hold-out set with wide intervals
One small anatomical structure
AI contours had modest geometric agreement
No prospective workflow or patient outcome
No external trial validation
Multi-source analysis · 2 linked sources. Primary analysed source date: 28 October 2026. Earliest linked supporting source: 29 May 2026.
Read the full report · 3 min The evidence, methods, implications and limitations
The model was used as a second measurement, not a replacement contour
Radiotherapy trials ask experts to check whether participating centres have drawn critical anatomy consistently. In this phase II prostate-cancer trial, the target structure was the internal pudendal artery, a small vessel whose contour varied substantially between clinicians. Of 120 enrolled patients, 107 plans from seven centres had the imaging, clinician contour and expert quality score needed for analysis.
The researchers generated an AI contour, measured its difference from the clinician's contour in 14 ways, and trained a classifier to flag likely protocol deviations. The AI outline was not especially close to the human one: mean Dice overlap was 0.446 and the 95th-percentile Hausdorff distance was 14.23. The useful signal came from the pattern of disagreements, not from treating the automated outline as clinically acceptable.[1][2]
A clean miss rate in one small test is not a deployment result
Across the 27-case hold-out set, the classifier reached an AUROC of 0.836, with a wide 95% confidence interval of 0.552 to 0.949. It produced no false negatives and six false positives. That means it preserved expert review for every poor contour in this test, but also asked reviewers to inspect acceptable work.
The denominator that matters is small: only a handful of hold-out cases were deviations, reflected in the sensitivity interval stretching down to 56.6%. The workflow was retrospective and tied to one vessel, one protocol and one expert reference process. A prospective trial would need to show that review time falls without missed errors, new bias or worse treatment decisions before clinics could rely on it.[1][2]
The distinction that matters
How the proposed triage works
The AI output becomes a comparison signal that helps decide which human contours still need expert attention.
Compare
Generate an AI contour beside the clinician contour
14 geometric disagreement measures
Triage
Classify likely protocol deviations
All hold-out deviations flagged; six extra reviews
Verify
Expert checks the selected cases
Prospective workload and safety evidence still needed