When the current symptoms were identical, did a psychiatric history change AI hospital advice?
In one synthetic vignette repeated 20,000 times, adding a prior schizophrenia diagnosis raised continued-hospitalisation recommendations from 4.7% to 22.3% when the current evidence supported discharge. No patients or clinical tools were tested.
The short version
- What happened
- Five general-purpose models answered four versions of one synthetic psychiatric vignette 1,000 times each, producing 20,000 responses.
- Who it affects
- If record history exerts excessive influence, a person whose present condition has improved could face more restrictive advice; this study identifies a testable risk, not an observed clinical event.
- What remains unproven
- Unreviewed preprint
Keep in mind
Unreviewed preprint
One synthetic vignette and one prompt
No patients, clinicians or clinical tool
One described White male patient
Repeated calls are not independent cases
Researcher-defined references are not clinical ground truth
Possible provider-side model changes
Source analysis · 1 linked source. Primary analysed source date: 26 October 2026.
Read the full report · 4 min The evidence, methods, implications and limitations
One line in the record changed the models’ recommendations
An unreviewed counterfactual experiment found that five general-purpose language models treated otherwise identical current evidence differently after a prior schizophrenia diagnosis was added to a synthetic medical record. When the vignette said the patient’s voices had ceased—an experimental condition prespecified to support discharge—the share recommending continued hospitalisation rose from 4.7% without the diagnosis to 22.3% with it. The pooled risk difference was 17.5 percentage points, with a 95% confidence interval of 16.2 to 18.8.
Researchers created four versions of one vignette by crossing current evidence—voices ceased or persisted—with diagnostic history—no prior diagnosis stated or schizophrenia recorded. GPT-5.6 Sol, Claude Sonnet 5, Gemini 3.6 Flash, Grok 4.6 and Mistral Large 2512 each completed 1,000 stateless calls per condition. That produced 20,000 outputs, but it did not involve 20,000 patients or 20,000 distinct clinical situations.[1]
The label mattered most where the current evidence supported discharge
All five models shifted toward continued hospitalisation in the discharge-supporting condition; model-specific risk differences ranged from 10.4 to 34.4 percentage points. Reference concordance fell from 95.3% to 77.7% while the models’ stated confidence increased. When voices persisted and the experimental reference supported continued hospitalisation, adding the diagnosis produced a smaller rise, from 92.6% to 97.0%.
Those results demonstrate label sensitivity, not necessarily an error mechanism or human-like stigma. Psychiatric history can contain relevant prognostic information, and the study cannot decide how much weight would be clinically appropriate. The prompts instructed general-purpose systems to act as psychiatrists, even though none was designed or validated for disposition decisions. Confidence was self-reported text, not a calibrated probability.[1]
The distinction that matters
What changed in the controlled vignette
The experiment altered diagnostic history and current symptoms separately while keeping all other text fixed.
Current evidence
Voices ceased or persisted
Experimental discharge reference changed
Past record
Schizophrenia diagnosis added or omitted
Current facts stayed identical
Measured output
Model chose discharge or hospitalisation
Label sensitivity, not patient outcome
A controlled stress test cannot establish clinical safety
The design used one White male vignette, one prompt and one prior diagnosis. The four versions isolated the selected record change but cannot show whether the pattern extends to other people, diagnoses, questions or clinical environments. Provider-side model changes during eight days of API calls cannot be excluded. The prespecified discharge categories were research references, not judgments from a clinician panel or observed patient outcomes. The study used no patient data and reports no specific grant funding.
Before any system influences psychiatric disposition, it should be tested on richer longitudinal cases that vary history and current presentation independently, with expert review and outcome evidence. Clinicians also need a way to see which record elements drove an answer. The useful conclusion today is narrow: in this repeated synthetic case, a historical label changed recommendations even when the current facts were held constant.[1]