What makes healthcare AI feel fair to patients?
In a survey of 149 UK respondents, the strongest concerns were decisions that could not be questioned, systems built without clinicians and incorrect health records. The convenience sample and hypothetical scenarios measure views, not the safety or fairness of a deployed tool.
The short version
- What happened
- Among 149 UK respondents, 89.9% said fairness would be harmed if patients or clinicians could not question an AI decision; 89.3% pointed to development without clinician input and 86.6% to incorrect electronic health-record data.
- Who it affects
- Patients want a practical route to question AI-influenced decisions and correct the health data behind them.
- What remains unproven
- The 149-person convenience sample is not a nationally representative estimate of UK patient opinion.
Keep in mind
The 149-person convenience sample is not a nationally representative estimate of UK patient opinion.
Responses concerned hypothetical vignettes rather than decisions made by a deployed system.
Cross-sectional attitudes cannot establish causal effects or predict behaviour under clinical pressure.
The study measured perceived fairness, not model performance, discrimination, safety or patient outcomes.
Source analysis · 1 linked source. Primary analysed source date: 13 October 2026.
Read the full report · 7 min The evidence, methods, implications and limitations
The answer: people want contestable decisions, clinical input and trustworthy data
For the 149 UK respondents in this cross-sectional survey, the clearest fairness warning was an AI decision that neither a patient nor a clinician could question. That condition was judged harmful to fairness by 89.9% of participants. Development without clinician input followed at 89.3%, and incorrect information in an electronic health record at 86.6%. The pattern was similar when people considered a patient-facing symptom checker and a clinician-facing tool intended to support cancer-treatment decisions. The result places practical accountability ahead of an abstract demand that a model simply be accurate.
This is evidence about perceptions, not proof that one feature makes an algorithm objectively fair. No system made a real diagnosis or treatment recommendation, and the researchers did not compare error rates across patient groups. The study instead identifies conditions that could make adoption legitimate or unacceptable to people likely to be affected. Its most useful implication is that healthcare organisations should not treat fairness as a model metric alone: patients also care about who helped build the tool, whether the underlying record is correct, who can intervene and whether a decision can be challenged before it affects care.[1]
What the researchers asked
Participants were recruited in the United Kingdom through public and patient involvement and engagement networks. They considered two short scenarios spanning different relationships with AI. In one, a person interacted directly with a symptom checker. In the other, a clinician used an AI decision-support tool while considering cancer treatment. Respondents rated how 13 factors would affect the fairness of the resulting decision and could describe additional concerns in free text. The team reported descriptive statistics, used chi-square tests to compare attitudes, estimated associations with ordinal logistic regression and analysed the written responses thematically.
The paired vignettes were a useful design choice because they tested whether concern disappeared when a clinician sat between the system and the patient. It did not: the importance assigned to oversight, professional involvement and data quality was consistent across both settings. Free-text answers also broadened the structured list. Respondents raised accessibility, the difficulty of representing multiple long-term conditions, the need to maintain systems after launch and the possibility that automation could erode human empathy. Those themes show why predeployment consultation needs open questions rather than a checklist written solely by technical teams.[1]
What the percentages can and cannot say
The percentages are denominated by 149 survey respondents, not by the UK population. Recruitment through involvement networks may reach people who have more experience of research, health services or patient advocacy than a random household sample. Voluntary participation can also favour those with strong interest in digital health. The study is cross-sectional, so it records views at one point in time, and hypothetical choices may not predict how someone responds when ill, under time pressure or faced with a clinician's recommendation. Subgroup comparisons from a sample this size should be treated cautiously.
Nor did the survey ask whether a specific system was fair after showing audited performance, subgroup error rates or a working appeals process. Respondents were reacting to described factors, which is appropriate for eliciting priorities but cannot reveal the trade-offs they would accept in practice. A person might value human oversight and still choose an automated route if it shortened a dangerous wait; another might reject a marginally more accurate tool if there were no route to correct a record. Those decisions require service-specific evidence and deliberation, not a universal threshold inferred from this survey.[1]
What this means for patients and clinical services
For patients, contestability needs to become a visible part of the service. A health-AI interface should say what information was used, what the output means, who remains responsible and how to report an error or request human review. A clinician-facing tool needs the same protections even if the patient never sees its screen, because a recommendation can still shape access, prioritisation or treatment. Correcting an electronic record must also be possible before inaccurate data are repeatedly fed into later decisions. A nominal human in the loop is insufficient if workload or policy makes disagreement impractical.
For developers and NHS organisations, the findings support involving clinicians and patients across design, testing, procurement and post-market monitoring. That does not mean letting a small consultation group declare a system fair. Teams still need evidence of diagnostic or operational performance, subgroup analyses, accessibility testing, security review and incident reporting. They should also decide how a model will be updated when clinical guidance or patient populations change. The authors received support from Wellcome, the Daphne Jackson Trust and several National Institute for Health and Care Research programmes; they report that funders had no role in design, analysis or writing and declare no competing interests.[1]
A wider definition of fairness
Technical fairness research often asks whether error rates or benefits are distributed similarly across demographic groups. That remains essential, especially where historic care patterns have produced biased records. The survey adds a complementary layer: procedural fairness. People want a route to challenge an output, meaningful professional participation and attention to maintenance, accessibility and empathy. A system could satisfy one statistical parity measure and still feel unfair if nobody can explain who is accountable or correct an obvious mistake. Conversely, an appealing interface and clinician presence cannot excuse unequal performance.
The study is UK-specific, but the underlying governance questions travel across health systems. The practical answer will differ where clinicians are scarce, records are fragmented, or care is paid for directly. Human review may be valuable yet unavailable at the moment a remote service is used. That makes it important to test alternative safeguards rather than export a high-resource workflow unchanged. Local patient groups should help define acceptable escalation, access and explanation, while regulators and providers retain responsibility for measurable safety and discrimination risks.[1]
What would change the assessment
A nationally sampled study, deliberately including people with disabilities, limited English, low digital access and multiple long-term conditions, would show how widely these priorities are shared. Experiments could vary real design features—such as response time for human review, availability of an explanation, clinician sign-off and a record-correction route—and measure choices rather than intentions alone. Qualitative work during real deployments could then examine whether people actually use challenge mechanisms and whether those mechanisms alter care.
The strongest evidence would link procedural safeguards with audited outcomes. Researchers should measure who receives a recommendation, who contests it, how often it changes, how long review takes and whether error and access gaps narrow or widen. Patient-reported trust should be reported beside safety, effectiveness and equity, not used as a substitute for them. Until that evidence exists, the new survey provides a clear design agenda and a warning against unchallengeable automation; it does not certify any healthcare AI system as fair.[1]