An EHR model transferred across hospitals—but rare-disease alerts were mostly false
TRACER was pretrained on 4.5 million patient records and tested in more than two million patients from two held-out health systems. At 95% specificity it found useful risk signals, but fewer than 1% of pancreatic-cancer alerts preceded a diagnosis.
The short version
- What happened
- TRACER was pretrained on 4,514,400 records from four University of California health systems; disease-specific survival heads used another 501,307 records, and two other UC systems with more than two million patients were held out for external evaluation.
- Who it affects
- Clinicians could receive a manageable-looking number of alerts while still reviewing many patients who will not receive the predicted diagnosis. Patients need evidence that follow-up tests and earlier action improve health without avoidable anxiety or harm.
- What remains unproven
- Preprint not yet peer reviewed; retrospective evaluation within one university health network, despite independently operated sites.
Keep in mind
Preprint not yet peer reviewed; retrospective evaluation within one university health network, despite independently operated sites.
External cohorts exceeded two million patients in aggregate, but endpoint-specific eligibility and outcome frequencies varied.
Non-UC care, free text, imaging, pathology and genomics were absent; recorded diagnosis was the outcome.
Thresholds were estimated separately in each cohort, so transport of a fixed operational threshold remains untested.
The authors declared no competing interests; the paper did not list a separate funding statement.
Source analysis · 1 linked source. Primary analysed source date: 29 October 2026.
Read the full report · 3 min The evidence, methods, implications and limitations
External testing exposed the cost of a useful-looking score
A health-record model retained much of its ability to rank future disease risk after moving to two hospitals excluded from development, but clinically relevant alert performance varied sharply. Researchers pretrained TRACER on 4,514,400 patient sequences from four University of California systems and trained survival heads on a separate 501,307. They then evaluated 16 diagnoses at one, two and five years in more than two million patients from two other UC systems.
At thresholds targeting 95% specificity, one-year heart-failure alerts captured 33.6%–44.2% of future diagnoses and selected about 5.2%–5.7% of records. Chronic-kidney-disease alerts were 8.0–8.5 times as likely as an unselected record to precede diagnosis. Yet pancreatic-cancer alerts had positive predictive values of just 0.29%–0.80%: most flagged patients were not diagnosed within a year.[1]
A screening candidate, not a clinical decision rule
The study's strength is that it reports sensitivity, positive predictive value and case yield rather than relying only on AUROC. It also preserves natural disease frequencies and accounts for incomplete follow-up. The trade-off is concrete: raising specificity from 90% to 99% reduced one-year heart-failure sensitivity from 63.7% to 15.1% in the development-domain evaluation.
TRACER used coded diagnoses, procedures, medicines and laboratory results, not notes, imaging or genomics. Care outside UC was missing, diagnosis codes mark documentation rather than biological onset, and short-horizon predictions may detect an investigation already under way. Each hospital's alert threshold was estimated retrospectively. Prospective trials must test fixed thresholds, follow-up workload, equity, costs and patient outcomes before deployment.[1]
The distinction that matters
One threshold, very different clinical meaning
At nominal 95% specificity, base rates determined how often an alert preceded a diagnosis.
Heart failure
5.2%–5.7% of records flagged
33.6%–44.2% of future cases identified
Kidney disease
One-year risk alert
8.0–8.5× enrichment over baseline
Pancreatic cancer
Similar share of records flagged
Only 0.29%–0.80% positive predictive value