Beyond Simulations: What 20,000 Real Conversations Reveal About Mental Health AI Safety.

Caitlin Stamatis¹, Jonah Meyerhoff², Richard Zhang¹, Olivier Tieleman¹, Matteo Malgaroli³, Thomas Hull¹

¹Slingshot AI.

²Northwestern University Feinberg School of Medicine.

³New York University.

PubMed PMC DOI PDF

Abstract

Large language models (LLMs) are increasingly used for mental health, yet safety evaluations rely primarily on small, simulation-based benchmarks removed from real-world language. We replicate four published safety evaluations assessing suicide risk handling, harmful content generation, and jailbreak resistance for general-purpose frontier models and a purpose-built mental health AI. We then condu…

Create a free account to keep reading

Free members get 10 full research views every month across publications, clinical trials, FDA clearances, adverse events, and NIH grants. No credit card required.

Create Free Account Sign In

Want unlimited research access? See Pro plans

Data Accuracy Notice: Research intelligence on Health AI Central is aggregated from public sources (PubMed, ClinicalTrials.gov, FDA, NIH, CMS, and others) and refreshed nightly. Classifications and derived metrics are produced by automated methods described in our Methodology. We recommend verifying critical data points against the primary sources before making decisions.