Can an AI risk message raise flu vaccination?
Three preregistered US field trials found that telling algorithm-identified patients they were at high risk produced a small absolute rise in flu vaccination. Naming the algorithm or explaining its reasons did not add a measurable benefit.
The short version
- What happened
- Three preregistered randomized trials at a US health system generated 116,811 trial assignments across three flu seasons and included more than 90,000 unique patients; some people could appear in more than one season.
- Who it affects
- A brief high-risk message may prompt a small additional share of selected patients to receive a recommended vaccine.
- What remains unproven
- The three seasons produced 116,811 randomized assignments but more than 90,000 unique patients; some people could contribute in more than one season.
Keep in mind
The three seasons produced 116,811 randomized assignments but more than 90,000 unique patients; some people could contribute in more than one season.
All trials were conducted within one integrated US health system, limiting geographic, financing and delivery-system generalization.
The article tests communication after algorithmic selection; it does not independently establish the risk model's calibration, fairness or clinical validity.
Absolute vaccination gains were small, and a larger relative percentage in a low-baseline season should not be confused with a large individual effect.
No added vaccination effect from algorithm disclosure or explanation does not establish that transparency is unimportant for trust, consent, legality or higher-stakes clinical decisions.
Recorded vaccination is a behavioural endpoint, not direct evidence of fewer influenza infections, hospitalizations or deaths.
Multi-source analysis · 5 linked sources. Primary analysed source date: 5 October 2026. Earliest linked supporting source: 19 March 2020.
Read the full report · 9 min The evidence, methods, implications and limitations
The evidence comes from three real-world randomized trials
Gail Rosenbaum and colleagues tested a practical question rather than a laboratory attitude: when a health system's machine-learning model identifies people at high risk of influenza complications, does telling them that risk change whether they receive a flu vaccine? The team ran three preregistered parallel-group field trials at Geisinger, a health system serving patients in Pennsylvania and nearby areas, across the 2020–21, 2021–22 and 2022–23 flu seasons. Vaccination recorded in the health system was the primary behavioural outcome; it was not a survey intention or a model benchmark.
The published extended-data denominators total 116,811 randomized assignments: 35,201 in study 1, 42,481 in study 2 and 39,129 in study 3. The article reports more than 90,000 unique patients because people could remain eligible and be randomized again in a later season. That distinction matters. The trial unit within each season was a patient assignment, but the three-study total is not 116,811 different people. The repeated-season design strengthens replication across time while preventing a simple interpretation of the pooled assignment count as an independent headcount.[1][2][3][4]
Patients were randomized to different explanations of the same risk flag
The health system had partnered with Medial EarlySign to score the risk of influenza complications from coded electronic-health-record data. In the first registration, eligible patients were in the model's top 10% of predicted risk, were at least 17 years old and had no recorded vaccine contraindication. Patients were randomized to receive no extra study message, a message saying they were at high risk, a high-risk message saying their medical records had been analysed, or a message adding that an algorithm or machine-learning analysis had produced the result. Later trials expanded the message comparison and used five arms, including an ordinary vaccination reminder as an active control.
The messages were delivered through the channels already available to the health system, including letters, text messages and secure patient-portal messages. Study 1's registered design masked patients to the existence of alternative study arms and did not tell care providers which patients were assigned to which communication. Studies 2 and 3 each split roughly 42,000 and 39,000 assignments across five similarly sized arms. The design therefore isolates small wording differences inside a live outreach programme, but it cannot isolate every surrounding influence: the pandemic, seasonal vaccine campaigns, prior contact with the system and changing baseline uptake all differed across years.[1][2][3][4]
The high-risk message produced a small but measurable lift
Across the trials, telling patients that they were at high risk increased recorded vaccination. Relative to patients who received a simple reminder to get vaccinated, the high-risk message added 1.1–1.4 percentage points, equivalent to a 3.3–5.4% relative increase. Relative to patients who received no study message, the gain ranged from 1.7 to 3.5 percentage points, or 3.3–14.7% relative. The two percentage forms answer different questions: percentage points describe the change in the share vaccinated, while the relative percentage depends heavily on the control rate.
Baseline vaccination also varied sharply by season. The passive-control rates shown in the paper's extended data were 50.8% in study 1, 31.5% in study 2 and 23.8% in study 3. A similar absolute gain therefore becomes a larger relative percentage when the starting rate is lower. For a health system sending tens of thousands of low-cost messages, one or two additional vaccinations per hundred contacted people may still be operationally useful. It is not evidence that personalized risk communication overcomes all access barriers, distrust, contraindications or hesitancy, and it should not be advertised as a dramatic individual effect.[1]
Naming the algorithm and explaining it did not add a detectable benefit
The most directly AI-related result is a null comparison. Vaccination was similar whether the message disclosed that an algorithm helped determine the risk, described an analysis of medical records without the algorithm label, or supplied personalized reasons for the prediction. In this setting, patients were neither measurably discouraged by the algorithm disclosure nor more persuaded by it. The operative content appears to have been the risk signal itself: the health system believed the patient faced elevated danger from influenza complications.
A null result is not proof that explanations never matter. These were brief outreach messages for a familiar preventive action already recommended for most people, not a consent conversation about diagnosis, treatment or denial of care. Patients could ignore the message without delegating a clinical decision to software. The study was powered for practical differences around two percentage points; smaller effects, subgroup effects or changes in trust may remain undetected. It also did not compare technically detailed explanations, interactive contestability or disclosure of model uncertainty.[1]
The study measures vaccination, not model accuracy or clinical benefit
The primary endpoint is consequential because it records behaviour, but it leaves two separate questions unanswered. First, the trials do not validate the machine-learning risk model in this article. Eligibility depended on a previously developed model, and the intervention tests communication among people it flagged. If calibration, subgroup performance or data quality were poor, a successful message could still target the wrong people. Second, an increase in vaccination is not the same as a demonstrated reduction in influenza, hospitalization or death; those events are rarer and need much larger or separately powered analyses.
The study also took place within one integrated US health system. Its patients had known contact routes and electronic records rich enough to run the model. Effects may differ where vaccination is paid for differently, delivered outside the originating health system, recorded incompletely, or constrained by supply, appointment access and language. People with limited portal use, unstable addresses or distrust of record-based profiling may receive or interpret messages differently. The paper's competing-interest statement notes that Geisinger paid Medial EarlySign to generate the risk scores and reasons; the authors report no other competing interests.[1][2][3][4]
The practical lesson is to separate the nudge from the model
For health services, the result supports a narrow implementation claim: a concise, credible high-risk message can add a modest amount of vaccination uptake among patients already selected by an algorithm. Teams should evaluate the whole pipeline separately. Model performance determines who is selected; message delivery determines who sees the intervention; wording affects response; and clinical operations determine whether an interested person can actually obtain a vaccine. A positive final uptake effect cannot reveal which earlier stage failed for everyone else.
The intervention should therefore be monitored with absolute numbers and denominators. Systems need delivery rates by channel, vaccination capture inside and outside the network, model calibration across demographic and clinical groups, opt-out rates, complaints and missed high-risk patients. Because the article found no added uptake from naming the algorithm or listing reasons, disclosure should not be optimized solely as a persuasion tactic. Transparency, lawful data use, the ability to ask questions and a route to correct records remain governance duties even when they do not increase clicks or vaccinations.[1][2][5]
What would change the assessment
Confidence in transferability would rise with preregistered replication in other health systems and countries, including safety-net settings, different languages and services where vaccination data arrive from pharmacies and public clinics. Reporting effects for delivery channel, age, deprivation, race and ethnicity, prior vaccine behaviour and baseline model risk would show who benefits and who is missed. A design that randomizes both message and access support—for example, a scheduling link, transport help or a no-cost walk-in option—could distinguish informational barriers from practical ones.
The model side also needs contemporaneous external validation and fairness analysis, with calibration and error rates reported for every group used operationally. Longer follow-up should test laboratory-confirmed influenza and serious complications rather than assuming that a small uptake gain produces an equal clinical benefit. Until then, the best-supported conclusion is modest: communicating high algorithm-estimated risk raised vaccination a little in three US field trials, while algorithm disclosure and personalized reasons did not measurably add to that effect. The work supports careful experimentation with outreach, not a blanket case for AI-driven medical messaging.[1][2][3][4][5]