Aneurysm AI kept 78% accuracy—and still was not ready for the clinic
A peer-reviewed study tested five algorithms on 75 patients and a small outside cohort. Treatment prediction transferred best; rare-complication detection and institutional bias remained serious limits.
The short version
- What happened
- Five algorithms were trained on 75 complete cases from 434 patients identified at one Arizona centre between 2002 and 2023.
- Who it affects
- People facing aneurysm treatment need advice based on anatomy, rupture status, local expertise and their own risks; a model that reproduces one centre's history could narrow rather than improve that discussion.
- What remains unproven
- Retrospective complete-case cohort of 75; four recorded complications; selected external sample of 20 most complete cases; one centre supplied training data; no prospective clinical or patient-outcome evaluation.
Keep in mind
Retrospective complete-case cohort of 75; four recorded complications; selected external sample of 20 most complete cases; one centre supplied training data; no prospective clinical or patient-outcome evaluation.
Source analysis · 1 linked source. Primary analysed source date: 16 October 2026.
Read the full report · 4 min The evidence, methods, implications and limitations
The outside test preserved one headline result
A machine-learning model predicted whether people with posterior communicating artery aneurysms received open surgery or endovascular treatment with 78.7% accuracy in repeated cross-validation and 78.1% accuracy at a second US centre. That close result is more informative than another high score from the training hospital alone. Yet it measures agreement with clinicians' past treatment choices—not whether the chosen procedure was best for an individual patient.
Researchers identified 434 patients treated at Barrow Neurological Institute in Arizona from 2002 to 2023, then retained 75 with complete medical, operative and imaging records. Fifty-nine had open surgery and 16 received endovascular treatment. Five algorithms were compared across seven targets, including treatment type, intraoperative rupture, complications, angiographic obliteration and discharge disability. Models used repeated stratified five-fold cross-validation, and the primary external analysis selected the 20 most complete cases from a Louisville cohort; endpoint-specific results used between 20 and 28 patients.[1]
Accuracy concealed the rare event the model missed
Only four of the 75 training patients had a recorded complication. A random forest therefore reached 94.7% accuracy for complications while detecting none of the positive cases: sensitivity was 0%. After class balancing, minority-class recall improved across several rare targets, but the small denominators leave wide uncertainty. The paper itself says external complication performance fell to 70.3% and treats the outside metrics as directional rather than precise evidence for deployment.
The training centre's practice pattern also shaped what the model learned. Seventy-nine per cent of its patients had open surgery, a distribution the authors say does not reflect most centres. The 21-year period spanned major changes in endovascular options, and complete-case selection reduced 434 identified records to 75. Some feature-selection descriptions also include downstream variables such as final disability, length of stay and rehaemorrhage, which weakens any claim that the full pipeline is a purely preoperative decision aid even when the immediate target was excluded.[1]
The distinction that matters
Why the same accuracy number can mean different things
A useful clinical model must predict the event that matters, transfer to another hospital and improve a real decision.
Treatment
Predict past procedure choice
78.1% outside accuracy
Complications
Find four rare training events
94.7% accuracy, 0% sensitivity
Clinical use
Guide care prospectively
Not tested
The model is a prototype, not a treatment recommender
External validation varied by target: treatment type transferred best, while procedure subtype and complication results were weaker. The study did not compare AI-assisted decisions with standard multidisciplinary care, test prospective use, measure whether recommendations improve disability or survival, or establish performance across hospitals with different treatment philosophies. The authors describe the models as centre-contextual prototypes and call for prospective, multi-institutional cohorts of 200 to 500 patients.
The article is a peer-reviewed accepted version and may still receive editorial corrections. The authors declared no personal, financial or institutional interest in the drugs, materials or devices described. For patients, the safe interpretation is narrow: external testing found some signal in routine clinical variables, but not a validated answer to whether clipping, coiling or another approach is right. Confidence would require larger representative cohorts, strictly pre-treatment inputs, prespecified outcomes, uncertainty intervals and a prospective comparison of clinical decisions and patient outcomes.[1]