Can AI prioritise cognitive assessment in sleep apnoea?
A random-forest model separated concurrent mild cognitive impairment in an external Chinese cohort, but overestimated probabilities and was evaluated almost entirely in men. It is not a diagnostic or prognostic tool.
The short version
- What happened
- Development used 280 Changchun patients, algorithm selection used another 120, and external validation used 192 patients from Shenzhen.
- Who it affects
- Some patients with sleep apnoea could be prioritised for formal cognitive assessment using information already collected in clinic.
- What remains unproven
- This cross-sectional model identifies concurrent MCI classification and cannot predict future cognitive decline or dementia.
Keep in mind
This cross-sectional model identifies concurrent MCI classification and cannot predict future cognitive decline or dementia.
The two cohorts came from Chinese sleep-medicine settings; international and language transportability is unknown.
Performance evidence came almost entirely from male patients, so validity in women is unestablished.
External calibration showed systematic probability overestimation despite strong discrimination.
Feature selection was performed once before repeated cross-validation rather than nested inside every resampling fold.
No AI-assisted referral pathway, clinician behaviour, treatment decision or patient outcome was tested.
Source analysis · 1 linked source. Primary analysed source date: 11 October 2026.
Read the full report · 6 min The evidence, methods, implications and limitations
The answer: the model separated groups well, but its raw risk numbers did not travel cleanly
In an independent Shenzhen sample, the selected random-forest model achieved a receiver-operating-characteristic area under the curve of 0.957 and a precision-recall area of 0.809 for concurrent mild cognitive impairment among people with obstructive sleep apnoea. Those figures indicate strong ranking discrimination in the study population. They do not mean that a predicted probability can be used unchanged in another clinic.
External calibration showed the problem: an intercept of -0.616 and slope of 0.701 indicated systematic overestimation and predictions that were too extreme. The authors explicitly position the model as a way to prioritise formal cognitive assessment, not diagnose MCI. It did not predict who would later deteriorate, compare an AI-assisted pathway with ordinary care, or show that referral decisions, treatment or quality of life improved.[1]
Who was included and how MCI was defined
The multicentre cross-sectional study included 400 eligible patients with obstructive sleep apnoea in Changchun. Researchers allocated 280 to training and 120 to an internal hold-out used to select the algorithm. A separate group of 192 patients from Shenzhen provided external validation. MCI was classified in 93 of 280 training participants, 38 of 120 internal participants and 57 of 192 external participants—roughly 30% to 33% in each group.
The outcome combined reported cognitive decline, objective evidence, largely preserved independence, exclusion of dementia and assessment-team consensus with physician participation. That is stronger than applying a single questionnaire cutoff, but it also makes the label dependent on a clinical process that other sites may not reproduce exactly. The paper says the retrospective operational specification cannot fully reconstruct the original case-report form from scale scores alone.[1]
How the model was built and selected
The investigators began with 27 candidate predictors, removed redundancy to leave 24 and used agreement between LASSO and Boruta to retain nine. The final variables were coffee consumption, physical activity, apnoea-hypopnoea index, mean oxygen saturation, depression and anxiety scores, daytime sleepiness, perceived stress and sleep quality. Eight algorithms were tuned with repeated ten-fold cross-validation, using SMOTE only inside training folds and precision-recall area as the primary metric.
The selected random forest used a threshold of 0.3228 derived from out-of-fold training predictions and locked before validation. Feature selection, however, was performed once on the complete training set rather than nested within every resampling fold. That choice may make the cross-validation stage more optimistic. The separate Shenzhen cohort is therefore important, though it validates one Chinese setting against another and does not establish transportability to different languages, health systems or patient mixes.[1]
The headline metrics need calibration and denominator context
In the 120-person internal hold-out used for algorithm selection, the random forest reached ROC-AUC 0.953, PR-AUC 0.936 and a Brier score of 0.060. Because that hold-out helped choose the algorithm, the paper appropriately warns that these are selection-conditioned results rather than an unbiased estimate of generalisation. In Shenzhen, ROC-AUC was 0.957, PR-AUC 0.809 and Brier score 0.051.
A high AUC means the model usually ranked a participant classified with MCI above one without it; it does not choose the right referral threshold or guarantee an accurate individual probability. Calibration matters when clinicians explain risk and allocate appointments. A model that overstates probability may send too many people for assessment, increase anxiety and divert capacity, even while its ranking metric remains impressive. Local calibration and capacity-aware threshold testing are necessary before any prospective use.[1]
What this could mean for sleep clinics and patients
Sleep clinics already collect several of the model's inputs, so a carefully evaluated score could help identify people who merit a formal cognitive review rather than waiting for concerns to become obvious. The likely benefit would be prioritisation, not automated diagnosis. Clinicians would still need to investigate medication, mood, education, language, hearing, vascular risk, sleep treatment and other explanations for performance on cognitive tests.
The model should not be applied to women on the strength of this paper because performance evidence came almost entirely from men. Nor should patients read coffee, exercise or mood variables as causal instructions: the cross-sectional model associates current characteristics with a current classification. It cannot show that changing one input prevents cognitive decline. Any clinical pathway should disclose uncertainty, avoid stigmatising labels and preserve access to a full assessment when symptoms conflict with the score.[1]
Limits, funding and evidence that would change the assessment
The study is cross-sectional, uses 592 participants across two Chinese centres and does not follow future cognitive outcomes. The external sample is independent but geographically and clinically related to development. Female representation was too small to support use in women. The once-only feature-selection step was not nested within resampling, and the selected predictors include self-reported lifestyle and symptom measures that can vary by culture, translation and time.
Funding came from Chinese national, provincial, university and education-development programmes. The paper reports ethics approvals at both participating centres. Confidence would rise with prospective validation in balanced-sex, multilingual and international cohorts; prespecified recalibration; comparison with a simpler clinical score; and decision-curve analysis tied to real appointment capacity. A trial showing earlier appropriate assessment without excessive false referrals, anxiety or unequal access would be needed to demonstrate patient benefit.[1]