A medical AI can hide its training data—and still reveal which patients were in it
Across 7,789 models trained on six public clinical datasets, average privacy scores concealed much higher risk for particular patients. A four-model screen found many high-risk records before full training, but it is not a privacy guarantee.
The short version
- What happened
- Researchers trained 7,789 models on six public datasets covering chest radiographs, dermoscopy, electrocardiograms and MRI; each patient was included in half of the shadow models used for the audit.
- Who it affects
- Patients whose scans or signals help train a model may face unequal privacy risk even when the dataset itself is never released. Data stewards need patient-level checks and clear protections before sharing or deploying trained models.
- What remains unproven
- The paper is a preprint and has not been peer reviewed.
Keep in mind
The paper is a preprint and has not been peer reviewed.
All six datasets were public research collections; performance in independent hospital pipelines is unknown.
The 0.7 attack-AUC threshold is operational rather than a validated boundary for privacy harm, and the screen does not establish that unflagged patients are safe.
The smallest cohort's top 5% contained 27 patients, and demographic variables were incomplete across datasets.
Source analysis · 1 linked source. Primary analysed source date: 21 October 2026.
Read the full report · 3 min The evidence, methods, implications and limitations
An average score hid who was most exposed
For each training recipe, 32–256 shadow models were trained on random halves of a patient cohort. The attack compared confidence from models that had and had not seen each record. A patient's score was the highest result across their records. Stronger weight decay reduced the number crossing 0.7 by as much as 2.95-fold, yet the ranking of the most vulnerable patients often persisted across recipes, architectures and initialisations.[1]
The proposed screen is triage, not protection
DataConfessor trains four small proxy models, scores records only with proxies that did not train on them and flags the highest 20% by atypicality. It recovered most of the reference audit's top 5% in 1.4–7.4 GPU-minutes. It performed less well on some transferred recipes, and a mitigation based on self-distillation lowered average scores without establishing protection at a low false-positive rate.[1]
The distinction that matters
Three layers of medical-model privacy evidence
The experiment separates a pooled model score from patient-level risk and preventive screening.
Cohort average
Pool confidence across records
Can conceal a high-risk tail
Patient audit
Compare models with and without each patient
Ranks membership-inference vulnerability
Pre-training screen
Flag atypical records with four proxies
Prioritises review; does not guarantee safety
Hospitals still need formal privacy controls
The study makes a practical case for checking patient-level risk during data preparation instead of relying on a reassuring cohort average. It does not show that an attacker identified real patients in a deployed hospital system. The authors say screening should complement—not replace—formal approaches such as differential privacy, access controls and deployment-specific auditing.[1]