Can a childhood-pneumonia risk model travel between hospitals?
A retrospective model trained on 5,546 children reached AUCs of 0.774 and 0.807 in two external Chinese cohorts. Different baseline risks shifted calibration, so local adjustment—not simple copying—remains necessary.
The short version
- What happened
- The retrospective study included 6,587 children treated across four Chinese medical centres between 2019 and 2023.
- Who it affects
- Children and families need timely escalation without avoidable alarm; that balance depends on local calibration and prospective workflow evidence.
- What remains unproven
- Retrospective records from four Chinese centres
Keep in mind
Retrospective records from four Chinese centres
No prospective deployment or clinician-impact study
External baseline-risk differences required recalibration
SHAP identifies model contributions, not causal drivers
No measured improvement in treatment, complications, hospital stay or recovery
Source analysis · 1 linked source. Primary analysed source date: 26 October 2026.
Read the full report · 3 min The evidence, methods, implications and limitations
Two hospitals developed the model and two tested it
Researchers assembled records for 6,587 children with Mycoplasma pneumoniae pneumonia across four medical centres from 2019 to 2023. Data from two centres formed a 5,546-child derivation cohort. Six machine-learning approaches were compared, with an untouched internal hold-out set used before the preferred random-forest model was taken to two other hospitals containing 696 and 345 children.
The model’s area under the receiver-operating-characteristic curve was 0.809 internally. External AUCs were 0.774 and 0.807. That consistency is more informative than another split of the development data because the external hospitals can differ in referral severity, testing, treatment timing and outcome prevalence. AUC nevertheless measures ranking—not whether the predicted percentages are correct or whether acting on them improves care.[1]
Calibration moved when baseline risk changed
The external populations had different rates of refractory disease, and calibration drift followed. The researchers improved calibration using a local intercept adjustment. This is an important deployment lesson: a model may preserve its ability to rank higher- and lower-risk children while systematically over- or underestimating the absolute risk in a new hospital.
SHAP analysis identified fever duration before admission, neutrophil percentage and age as the leading contributors. Those variables are clinically interpretable, but an explanation of the model’s calculation is not proof that the relationship is causal. Fever duration can also reflect when families seek care and when hospitals admit, rather than biology alone.[1]
The distinction that matters
Discrimination and calibration answer different questions
A model can rank patients consistently while giving the wrong absolute risk.
Ranking
External AUCs 0.774 and 0.807
Higher-risk children tended to rank higher
Probability
Baseline prevalence differed
Calibration drift appeared
Local step
Adjust the intercept and retest
Fit improved, clinical benefit still unknown
A transportable score still needs a clinical-use test
For paediatric teams, early risk stratification could help target monitoring and review. It could also create unnecessary escalation if a locally miscalibrated threshold labels too many children as high risk. Before use, each hospital would need to check prevalence, missing-data handling, alert volume, subgroup performance and whether clinicians can respond without displacing established assessment.
The study was retrospective and conducted within one national health context. It did not deploy the model prospectively, compare model-guided care with usual practice or measure antibiotics, length of stay, complications and recovery. The assessment would change with preregistered prospective validation, transparent decision thresholds and a controlled study showing that use improves care rather than prediction metrics alone.[1]