Can you trust citations in Google’s AI health answers? A 3,611-question preprint found weak grounding
Google showed an AI Overview for 96.4% of the benchmark questions, yet many factual statements lacked citations and only about half of scorable cited claims were supported. This was one US-English snapshot, not a test of patient outcomes or today’s product.
The short version
- What happened
- Researchers submitted 3,611 English HealthBench questions to Google Search in a US locale on 16–18 September 2026; 3,481, or 96.4%, returned an AI Overview.
- Who it affects
- People may read a cited AI health answer as more authoritative than an uncited one. This study says the presence of citations alone is not enough to establish support, accuracy or completeness; urgent and personal decisions still need qualified clinical advice.
- What remains unproven
- Unreviewed preprint
Keep in mind
Unreviewed preprint
One Google product snapshot
US-English locale
September 2026 collection
Automated source and answer grading
Limited human validation samples
Unreadable cited pages excluded
Complex benchmark questions
No patient behaviour or clinical outcomes
No causal test of how citations were generated
No separate funding or competing-interests statement
Source analysis · 2 linked sources. Primary analysed source date: 29 October 2026.
Read the full report · 4 min The evidence, methods, implications and limitations
Citations were common, but support was inconsistent
A new preprint found that Google frequently produced AI-generated health summaries without consistently tying their factual claims to supporting pages in this snapshot. Researchers sent 3,611 realistic, problem-focused HealthBench questions to Google Search from 16 to 18 September 2026, fixing the locale to US English. One retrieval failed; 3,481 of the remaining 3,610 questions returned an AI Overview. Of 3,474 usable overviews, 327 provided no source citation at all.
The team split the answers into 24,060 statements. One in five was classified as factual but uncited. Of 12,175 cited statements that the automated system could score against a linked page, 47.1% were supported. After adjusting for the classifier’s measured sensitivity and specificity, the estimate rose to 56.7%, but its 95% interval stretched from 45.3% to 68.6%. That uncertainty matters: neither number is a direct human audit of every claim.[1][2]
A large benchmark was paired with automated checks and physician rubrics
The questions came from 5,000 HealthBench conversations, filtered to 3,611 English health queries. The researchers captured each overview, all its citations and Google’s five highest traditional results. Their source-quality script applied four JAMA disclosure criteria and two HONcode-derived checks; on 174 human judgments it agreed 84% of the time, with Cohen’s kappa of 0.68. The grounding classifier was validated on 100 statements, reaching 81% agreement and kappa 0.61.
For answer quality, 41,180 physician-written rubric criteria were graded by GPT-4.1. On 6,200 physician grades from a random 3,000-example validation set, its macro-F1 was 0.707, compared with inter-physician agreement of 0.681. The overviews earned 38.2% of all available rubric points, including 54.2% for accuracy and 21.6% for completeness. These are fractions of detailed rubric points—not a claim that every answer was simply 38.2% correct. The cited pages also scored slightly lower than traditional results on four of six disclosure and authority checks, although every standardised difference was small and the largest absolute effect was 0.19.[1][2]
The distinction that matters
What the study measured—and what it did not
Three linked tests examine different parts of an AI health answer; none measures patient benefit or harm.
Coverage
3,611 health questions
3,481 returned an AI Overview
Grounding
24,060 extracted statements
Citations checked against linked pages
Answer quality
41,180 physician rubric criteria
Automated grading with physician validation
Not measured
Patients and clinicians
No behaviour, decisions or health outcomes
A citation icon should not be treated as a clinical guarantee
The study also found little relationship between citation grounding and rubric accuracy or completeness: Spearman’s rho was 0.01. Among 2,065 questions whose physician rubric anticipated a particular inaccurate or potentially harmful statement, 22.2% of overviews made at least one such statement. Yet the system referred 94.9% of 413 emergency-referral questions with the expected urgency. The evidence therefore describes uneven performance, not uniform failure.
This is an unreviewed snapshot of one changing product in one language and locale. HealthBench questions can be long and complex, although a shorter single-turn subset produced similar patterns. Automated graders, inaccessible pages and text-only extraction add error; 11.7% of cited pages and 14.0% of traditional results were excluded as unreadable. No patients, clicks, treatment decisions or health outcomes were studied. Independent replication on current versions, clinician-led claim review and user-outcome research would change the assessment. The manuscript gives no separate funding or competing-interests statement.[1][2]