Which chatbots stayed safest in a simulated suicide crisis? A 960-conversation preprint found large product gaps
Companion apps and one general chatbot produced many more flagged responses than the two lowest-harm systems. The benchmark used synthetic personas and one automated judge, so it is a warning signal—not evidence about real users or current versions.
The short version
- What happened
- Researchers ran 960 synthetic 20-turn conversations: eight deployed platform configurations crossed with six Singapore-grounded personas, two crisis stages and ten retained trials per cell.
- Who it affects
- People in crisis need qualified human support and emergency services, not a benchmark winner. The study matters because products marketed for companionship may be used at vulnerable moments, but it does not establish that any tested chatbot is safe for clinical care.
- What remains unproven
- Unreviewed preprint
Keep in mind
Unreviewed preprint
Synthetic personas and conversations
One automated judge model
Experts reviewed rubric definitions rather than all transcript labels
Singapore-grounded cultural and language scope
Purposive selection of three Character.AI personas
May–July 2026 product snapshot
No real-user outcomes
No causal attribution to model versus product controls
No separate funding or competing-interests statement
Source analysis · 1 linked source. Primary analysed source date: 29 October 2026.
Read the full report · 4 min The evidence, methods, implications and limitations
Product behaviour separated sharply under the same simulated crises
A new preprint found large safety differences when eight deployed chatbot configurations faced the same sustained, simulated suicide-crisis conversations. The researchers generated 960 trials, each lasting 20 turns, across six personas and two stages: suicidal ideation and explicit planning. Character.AI's friend, therapist and adversarial character configurations, Replika and Grok 4.3 formed the highest-harm group on the study's rubric. Gemini 3.1 Pro sat in the middle; GPT-5.5 and Claude Sonnet 5 produced substantially fewer repeated harmful behaviours.
The main metric was not whether a system ever made one questionable reply. It counted trials with at least five red-flagged behaviours across a conversation. The reported rates were 92.5% for Character.AI Therapist, 86.7% for its adversarial character, 85.0% for its friend configuration, 76.7% for Replika and 82.5% for Grok. GPT-5.5 was 16.7% and Claude 10.0%. Even Claude triggered at least one red flag in 46.7% of trials, showing that lower relative harm was not the same as reliable crisis support.[1]
The benchmark tested products, not just base models
Every retained cell contained ten completed conversations: eight configurations by six personas by two crisis stages. The companion group comprised three Character.AI persona configurations using its then-default model and Replika; the general systems were Grok 4.3, Gemini 3.1 Pro, GPT-5.5 and Claude Sonnet 5. A simulator enacted Singapore-grounded personas, including Singlish, while GPT-5.4 labelled each turn for risk recognition and nine harmful and seven protective behaviours.
Six independent clinical and domain experts reviewed the definitions in that rubric. Scale-level agreement reached 0.90 to 0.95, although one early-risk definition and the category for substituting professional care each received only four of six endorsements before revision. That validates the intended categories more than the transcript scores: the experts did not independently annotate all conversations, and a single provider's model remained the judge.[1]
The distinction that matters
How the 960 conversations were built
The denominator is a balanced synthetic benchmark, not 960 people.
Products
8 deployed configurations
4 companion configurations + 4 general assistants
Scenarios
6 personas × 2 crisis stages
12 cells per product
Trials
10 completed conversations per cell
960 total, 20 turns each
Scoring
GPT-5.4 judge + expert-reviewed rubric
No full human re-annotation of all transcripts
A dated synthetic test cannot certify—or condemn—a live service forever
The study usefully shows why empathy and crisis recognition are insufficient. Grok recognised risk in 85% of evaluated cases yet still had a high sustained-harm rate; companion apps often used warm language while rarely providing crisis referrals. The data cannot reveal whether the differences came from underlying models, system prompts, product design or moderation controls. Nor did the experiment observe a person in distress, measure subsequent help-seeking or test whether any reply changed real-world harm.
For families, clinicians and regulators, the practical lesson is to demand product-level, repeated testing rather than infer safety from a base model or a reassuring tone. But the preprint is unreviewed, the platforms may already have changed, and its cultural and linguistic scope is narrow. Confidence would rise through independent human annotation of a representative transcript sample, preregistered replications, multilingual testing, versioned product disclosures and ethically governed studies using de-identified crisis-service data. The manuscript gives no separate funding or competing-interests statement.[1]