Can health systems evaluate generative AI?

The direct answer: the machinery exists, but it was built for more stable technologies

Health systems already know how to ask whether a technology works, costs less than alternatives and delivers value in routine care. The new comparative review finds that those systems do not yet give assessors a specific method for generative AI's defining problem: the same model can produce different answers to the same task, respond sharply to small prompt changes and change after an update. That gap matters because a favourable evaluation can influence reimbursement, coverage and procurement, while an incomplete one can mistake a strong average score for dependable performance in real care.

The paper does not show that every current decision is wrong or that a particular AI tool is unsafe. It maps written guidance, not decisions or patient outcomes. Its narrower conclusion is important enough: across six deliberately different health systems, the authors found established clinical and economic evidence requirements but no dedicated guidance for sampling variable model outputs, calculating how many outputs must be tested, or reassessing a system as its behaviour changes.[1]

What the researchers reviewed

The researchers used a policy-mapping method based on scoping-review principles and reported the search with PRISMA-ScR. They chose Canada, France, Germany, South Korea, the United Kingdom and the United States to cover different health-technology-assessment structures and because the team could work in English, French, German and Korean. The set spans a single-payer threshold model, comparative-benefit assessment, medico-economic evaluation, a federated system, conditional reimbursement and the US mixture of public coverage bodies and a private value-assessment organisation.

Repository searches identified 22,772 policy records: 15,504 from Canada, 241 from France, 3,410 from Germany, 39 from South Korea, 851 from the UK and 2,727 from the US. A supplementary PubMed search found 20,571 academic records. After screening and eligibility checks, 56 policy documents remained for qualitative synthesis. The searches were first run in early 2025 and updated on 1 November 2025, so the paper describes guidance available at that cut-off rather than every policy change since.

The team grouped relevant passages into four domains: efficacy and effectiveness, health economics, social and equity considerations, and uncertainty management. One author extracted themes and clustered them; three co-authors reviewed the clustering. The study therefore offers a structured cross-system comparison, but it is not a double-coded systematic review of every document and it did not grade the quality or legal force of the included guidance.[1]

Where the six systems already agree

All six systems distinguish technical or clinical efficacy from effectiveness in real practice. Randomised controlled trials remain preferred for higher-risk technologies that can change diagnosis or treatment, while several systems allow alternatives when software evolves too quickly or a randomised design is impractical. France explicitly permits pragmatic trials and target-trial emulation in relevant circumstances; the UK and US recognise observational and real-world evidence; Germany, South Korea and the US have conditional pathways that can tie access to further evidence collection.

Economic evaluation also has a familiar core. Cost-effectiveness and cost-utility analysis remain central, often using quality-adjusted life years. Some frameworks allow cost-consequence approaches or adaptive inputs when one summary measure misses workflow or caregiver effects. That flexibility is useful for software that saves time or changes how care is organised, but the review found little operational direction for modelling a model whose performance distribution can itself move.

Social and equity assessment was less consistent. Some systems ask developers to test performance across demographic groups or use qualitative evidence, while South Korea had no specific social or equity guidance in the reviewed set. A generic requirement to consider bias is not the same as a protocol for testing prompt, user, language and update effects across groups. The practical task is to specify which populations, tasks and deployment settings must be represented—and what happens when performance falls outside an agreed range.[1]

The generative-AI mismatch is about variability, not just opacity

Conventional assessment often assumes a technology has a bounded, reasonably stable set of outputs. Generative models sample from a distribution. Repeating an apparently identical prompt can yield different text, and rephrasing the prompt or changing the user can shift the result again. A single test answer is therefore not a sufficient unit of evidence. Assessors need rules for repeated sampling, clustered observations and uncertainty intervals that reflect both patients and model outputs.

Updates create a second problem. A clinical evaluation may describe one model version while the deployed service changes through fine-tuning, vendor updates or altered system prompts. The authors point to predetermined change-control plans in medical-device regulation as a possible template, but assessment bodies still need triggers for re-testing, fresh economic modelling and withdrawal or restriction. Conditional reimbursement can help only if the required real-world evidence, review period and action thresholds are specified in advance.

The review proposes directions rather than a finished checklist: sample multiple outputs; test sensitivity to prompts and users; use updating methods when performance evolves; define change controls; and make post-market evidence obligations explicit. Those suggestions are plausible, but this study did not compare them experimentally. A framework that adds repeated tests without measuring clinically important outcomes could create paperwork without resolving the decision problem.[1]

What this means for patients, clinicians, payers and developers

For patients, the gap can work in both directions. A health system may over-accept an impressive validation result that does not survive ordinary prompts, languages or model updates. It may also under-value a useful tool when existing measures overlook time saved, access improved or caregiver burden reduced. Neither risk justifies bypassing clinical evidence. Workflow benefit belongs alongside, not in place of, accuracy, safety, equity and patient outcomes.

Clinicians need to know which version was evaluated, what inputs were tested and when the system should not be trusted. Payers need contracts that preserve audit access and connect material changes to reassessment. Developers should expect a stronger evidence package than a single benchmark: prespecified tasks, repeated outputs, relevant comparators, subgroup reporting, calibrated thresholds and monitoring after deployment. Procurement teams should not treat a regulator's market authorisation, a technology-assessment recommendation and a local implementation decision as interchangeable approvals.[1]

The evidence limits and the commercial context

The six jurisdictions were purposively selected, not sampled to represent the world. Australia and the Netherlands were omitted, and many lower- and middle-income health systems have different capacity, financing and data constraints. The corpus mixed binding rules, methods manuals and non-binding frameworks. Treating them together helps map what is written, but a recommendation in a policy paper does not carry the same force as statute or a reimbursement rule.

The authors did not formally assess document quality. One researcher conducted the supplementary academic search, and one extracted and clustered themes before post-hoc review by three colleagues. The public documents and extraction sheet support scrutiny, but independent coding and a prospective update would strengthen confidence. The policy cut-off also means the study cannot establish whether agencies have since adopted unpublished practices or newer guidance.

Roche Diagnostics funded the underlying project. Four co-authors were Roche Diagnostics employees, and LSE employed six authors while receiving Roche funding for the analysis. The paper says the academic team retained editorial control and that no specific funding was received for writing this article. Those disclosures do not invalidate the results, but they matter because diagnostics companies have a commercial interest in how evidence and reimbursement frameworks evolve.[1]

What would change the assessment

The next step is not another high-level principle. Assessment bodies should publish operational protocols and test them on real products: how many outputs to sample, how to vary prompts and users, how to define a material model change, which subgroup and setting checks are mandatory, and which failures pause or reverse coverage. Public case studies should show how those rules change a decision compared with current practice.

Confidence would increase through a broader, independently coded update covering more regions and every new policy after November 2025. Comparative evaluations could then examine whether generative-AI-specific guidance improves the quality, speed and consistency of decisions without creating avoidable barriers. The decisive evidence is prospective: fewer unsafe deployments and missed benefits, better-calibrated reimbursement, transparent reassessment after updates and measurable patient outcomes. Until then, the paper identifies a credible gap, not proof that one proposed remedy has solved it.[1]