How safe was ChatGPT health advice in four Global South countries?
An analysis of 15,555 donated health conversations rated about one in five high-risk. The estimate comes from an unreviewed convenience sample and an LLM judge, not clinician review or measured patient harm.
The short version
- What happened
- Researchers identified 15,555 health conversations in complete ChatGPT histories donated by 1,252 users in India, Nigeria, Brazil and Pakistan.
- Who it affects
- People should treat chatbot health guidance as general information and seek qualified care for symptoms, medicines, diagnosis, mental-health crises or urgent decisions.
- What remains unproven
- This is an unreviewed preprint using a self-selected convenience sample, not a representative survey.
Keep in mind
This is an unreviewed preprint using a self-selected convenience sample, not a representative survey.
Safety labels came from an LLM judge; clinicians did not establish sensitivity or correctness across the corpus.
The study measured potential risk, not whether advice was followed or caused an outcome.
Context-aware results apply only to the subset with earlier profile information.
Model-version comparisons are observational and confounded by time, user choice and query composition.
Source analysis · 1 linked source. Primary analysed source date: 13 October 2026.
Read the full report · 7 min The evidence, methods, implications and limitations
The answer: the audit found substantial risk, but not a country prevalence rate
In a consent-based corpus of real ChatGPT histories, an automated safety audit rated roughly one in five health conversations high-risk. The rate was 21.5% in India, 20.6% in Nigeria, 22.3% in Brazil and 26.3% in Pakistan. Questions about symptoms, medicines and ongoing conditions were much more likely to be flagged than general health information.
That is consequential evidence because it examines what people actually asked rather than clinician-written vignettes. It is not proof that one in five ChatGPT health exchanges in any country caused harm. The users were a self-selected crowdworker sample, the safety judge was another language model, and the researchers did not observe whether anyone followed the advice or experienced an adverse outcome.
For people seeking care, the practical message is that fluent advice should not substitute for diagnosis, prescribing or urgent triage. For developers and regulators, the study shows why a response that looks acceptable in isolation may become unsafe when the system already knows that a user has a chronic condition, takes medication or is in mental distress.[1]
The dataset linked conversations to the people who donated them
Researchers recruited 1,252 active ChatGPT users through Clickworker in India, Nigeria, Brazil and Pakistan between December 2025 and February 2026. Participants exported complete histories covering December 2022 to February 2026 and reported age and gender. A client-side process removed names, email addresses and other identifying information before upload.
The full corpus contained 202,590 conversations. Topic modelling identified candidate health clusters, after which 15,555 conversations from 754 users were treated as health-related. India contributed 6,343 health conversations, Brazil 4,657, Nigeria 2,976 and Pakistan 1,579. Median health conversations per contributing user ranged from four in India to seven in Brazil.
The denominator matters. A conversation is not a person, and users contributed different numbers of sessions. India supplied 44.5% of participants, while the age and gender mix differed sharply by country. The study therefore foregrounded effect sizes and cross-country consistency rather than treating every conversation as an independent population survey.[1]
Classification and safety scoring were largely automated
A GPT-4o-mini classifier assigned messages to a twelve-part health-intent taxonomy using the preceding five user-assistant turns. A blinded author check on 100 sampled messages found 82% agreement and Cohen's kappa of 0.78. That supports the intent labels, but one hundred checks are small beside 15,555 conversations and most disagreements occurred between adjacent categories.
Gemini 3.1 Pro Preview scored the last assistant message on likelihood of harm, potential severity and adequacy of safeguards using a published seven-point rubric. The conversation inherited its lowest message score, and the researchers defined three or below as high-risk. One author read 400 sampled conversations and could assign a concrete failure mode to all 128 the judge had flagged.
This process tests potential danger, not clinical correctness against a gold-standard diagnosis. The human audit supports the precision of flagged cases but does not estimate risky answers that the judge missed. No clinician panel reviewed a representative sample, and no medical record or outcome showed whether advice was accurate for a particular patient.[1]
Risk concentrated where users sought personal clinical guidance
General health information made up 28.3% to 48.9% of health conversations by country and was rated high-risk in 7.9% to 11.9% of cases. Condition-and-care questions were flagged in 39.8% to 47.7%, symptom questions in 33.4% to 40.3%, and emotional-wellbeing conversations in 22.7% to 34.3%.
At user level, 51.6% of Nigerian contributors with health conversations and 58.3% to 61.7% in the other countries received at least one flagged response. This is different from the per-conversation rate: frequent users have more chances to encounter one low-scoring exchange, so it should not be read as the share of all citizens exposed to dangerous advice.
Among 128 flagged conversations, the most common coded failures were incorrect or harmful advice in 61 and missing professional referral in 56. Unsafe remedies, unsafe diagnosis and reinforcement of harmful behaviour or dependence each appeared in 32. Codes overlapped, so the counts do not add to one prevalence total.[1]
Earlier user context changed the safety assessment
For conversations preceded by health-relevant history, the researchers built a running profile from durable facts disclosed in earlier chats: chronic conditions, medication use, lifestyle, mental state and family circumstances. GPT-5-mini extracted the profile, and the same Gemini judge rescored the unchanged response against that context.
Adding context raised the high-risk share by 13 to 17 percentage points, and about one quarter of conversations rated safe in isolation became unsafe for the person asking. Fitness advice changed especially sharply when prior eating disorders, injuries or vulnerabilities were visible. Emotional-wellbeing advice also became riskier when earlier distress was considered.
Only 19% to 60% of health conversations by country had enough preceding profile information, and that subset was already riskier in context-blind scores. The rate cannot be extrapolated to every health conversation. It nevertheless exposes a design problem: a system may possess facts needed to avoid harm without using them in a safety check.[1]
Newer model versions were not shown to be safer
Among 14,420 conversations served by one identifiable model, high-risk ratings rose from GPT-3.5 through GPT-5.1 in each country. In 240 users who asked health questions of both GPT-5 and GPT-5.1, 19.0% of GPT-5 conversations and 27.8% of GPT-5.1 conversations were flagged.
The authors appropriately stop short of saying later models were intrinsically less safe. Model version is entangled with time, changing user behaviour and selection into mini variants. Correlations were statistically significant but weak, and GPT-5.2 estimates rested on only 30 to 282 conversations per country.
The defensible conclusion is narrower: observed risk did not reliably decline as versions changed. Release recency should not be treated as evidence of medical safety. Providers need version-specific monitoring on real use, with independently defined clinical thresholds and transparent changes to safeguards.[1]
Limits and what would change the assessment
Participants were paid crowdworkers willing to donate complete histories. They were active, digitally literate users, not representative national samples. India and Pakistan were young and male-skewed, older groups were small, and the analysis did not cover clinical severity, income, rurality, language proficiency or access to care.
Confidence would rise with a preregistered clinician audit of a stratified sample that estimates both false alarms and missed hazards. Sampling should include rural, older and lower-literacy users, local languages and people with limited access to care, while preserving consent and data minimisation.
The decisive next evidence is behavioural and clinical: whether users acted on advice, whether referral or triage was delayed and whether outcomes changed. Until then, this is an early post-deployment safety signal showing that authentic use and earlier context can reveal risks hidden by synthetic benchmarks, not a clinical error rate or proof of realised harm.[1]