Spine-surgery AI looked accurate—but extra imaging features added little

A locked model cleared a stronger test

A spine-surgery risk model performed strongly when tested prospectively, correctly classifying 100 of 108 patients with complete 12-month follow-up. At its prespecified threshold, it identified 29 of 34 poor outcomes and 71 of 74 good outcomes, producing five false negatives and three false positives. This is more credible than tuning a model after seeing the test cohort—but it is not yet evidence that using the prediction improves care.

Researchers developed the system on 665 retrospective patients: 532 for training and 133 for internal validation. They combined clinical variables and conventional imaging with 1,320 CT and MRI radiomic features and pre-generated 768-dimensional deep features from each imaging modality. The feature selection, model, hyperparameters, decision threshold and recalibration procedure were locked before any prospective data entered the pipeline.[1]

Discrimination was high; calibration was not

The prospective cohort was recruited consecutively between April 2024 and February 2025. Twelve of 120 recruits lacked complete follow-up, leaving 108 for analysis. A poor outcome meant residual pain above two on the visual analogue scale or disability above 20% on the Oswestry index at 12 months. The model reached 85.3% sensitivity, 95.9% specificity and 92.6% overall accuracy; ROC-AUC was 0.976.

Those numbers describe ranking and classification, not trustworthy individual probabilities. Before recalibration, the prospective Brier score was 0.101, the calibration intercept 2.629 and slope 3.180—well away from the ideal intercept of zero and slope of one. Training-derived recalibration improved the Brier score to 0.084 but left material miscalibration. A patient should not be told a numerical risk as though it were already reliable.

The expensive-looking feature stack also added limited discrimination beyond ordinary clinical and conventional-imaging information. That matters for hospitals weighing extraction, annotation and compute against a simpler reference model. More variables do not automatically create more useful decisions.[1]

The image pipeline cannot yet be fully audited

A further qualification sits upstream. The archived records described a ViT-Base image encoder, but the implementation, pretrained-weight source, output layer, input processing and any fine-tuning were not retained. The authors therefore could not fully reconstruct or independently audit how the deep features were created. That is a serious reproducibility gap for a model intended to inform clinical risk.

For surgeons and patients, the evidence check is encouraging but bounded: one centre, 108 prospective patients and only 34 poor outcomes leave wide uncertainty, particularly around calibration and subgroups. Twelve of the 120 prospectively recruited patients were excluded because follow-up was incomplete, which could matter if their recovery differed systematically. The model was not compared in a trial where clinicians acted on predictions, and no treatment, complication or recovery benefit was measured. Chinese public grants funded the work and no commercial conflict was declared. Confidence would rise with multicentre locked-model validation, a reproducible image pipeline, comparison against a simple clinical score and a prospective impact trial showing safer decisions or better recovery.[1]