ArticleNPJ digital medicine2023
Natural language processing system for rapid detection and intervention of mental health crisis chat messages.
Article in NPJ digital medicine, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 19 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
19 citing papers in PubMed, 1 synthesis or guideline pooled it, 33 citations in OpenAlex.
- The Role of Artificial Intelligence for Intimate Partner Violence Prevention: A Systematic Review.Journal of clinical nursing · 2026Pooled it
- Large language model-based triage to identify antiretroviral therapy adherence barriers and risk levels in patient messages.JAMIA open · 2026Article
- Conversational AI should fill the white space in mental health care, not replace humans.NPJ digital medicine · 2026Review
- Speech markers of psychological change following a psychedelic 5-MeO-DMT retreat.Journal of psychopharmacology (Oxford, England) · 2026Article
- Identifying suicide-related language in smartphone keyboard entries among high-risk adolescents.NPJ digital medicine · 2026Article
- Surfacing Suicidal Risk Through Simulated Social Interaction: Per-Person Language Model Agents as Communicative Stress Tests.medRxiv : the preprint server for health sciences · 2026Article
- Large language models for post-discharge follow-up in erectile dysfunction care: a narrative review.Translational andrology and urology · 2026Review
- In their own words: case studies of adolescent smartphone language preceding suicide-related hospitalizations.NPP - digital psychiatry and neuroscience · 2026Article
- Diagnostic accuracy of generative large language artificial intelligence models for the assessment of dental crowding.BMC oral health · 2025Article
- Operationalizing machine-assisted translation in healthcare.NPJ digital medicine · 2025Review
- Evaluating natural language processing derived linguistic features associated with current suicidal ideation, past attempts, and future suicidal behavior.Journal of psychiatric research · 2025Article
- Smartphone-based text obtained via passive sensing as it relates to direct suicide risk assessment.Psychological medicine · 2025Article
- The FAIIR conversational AI agent assistant for youth mental health service provision.NPJ digital medicine · 2025Article
- Artificial intelligence tools in supporting healthcare professionals for tailored patient care.NPJ digital medicine · 2025Article
- Predicting Satisfaction With Chat-Counseling at a 24/7 Chat Hotline for the Youth: Natural Language Processing Study.JMIR AI · 2025Article
- Natural language processing to identify suicidal ideation and anhedonia in major depressive disorder.BMC medical informatics and decision making · 2025Article
- Large Language Models Versus Expert Clinicians in Crisis Prediction Among Telemental Health Patients: Comparative Study.JMIR mental health · 2024Article
- Artificial Intelligence and Occupational Health and Safety, Benefits and Drawbacks.La Medicina del lavoro · 2024Article
- Examining Passively Collected Smartphone-Based Data in the Days Prior to Psychiatric Hospitalization for a Suicidal Crisis: Comparative Case Analysis.JMIR formative research · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
13 authors at 4 institutions in 2 countries.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Patients experiencing mental health crises often seek help through messaging-based platforms, but may face long wait times due to limited message triage capacity. Here we build and deploy a machine-learning-enabled system to improve response times to crisis messages in a large, national telehealth provider network. We train a two-stage natural language processing (NLP) system with key word filtering followed by logistic regression on 721 electronic medical record chat messages, of which 32% are potential crises (suicidal/homicidal ideation, domestic violence, or non-suicidal self-injury). Model performance is evaluated on a retrospective test set (4/1/21-4/1/22, N = 481) and a prospective test set (10/1/22-10/31/22, N = 102,471). In the retrospective test set, the model has an AUC of 0.82 (95% CI: 0.78-0.86), sensitivity of 0.99 (95% CI: 0.96-1.00), and PPV of 0.35 (95% CI: 0.309-0.4). In the prospective test set, the model has an AUC of 0.98 (95% CI: 0.966-0.984), sensitivity of 0.98 (95% CI: 0.96-0.99), and PPV of 0.66 (95% CI: 0.626-0.692). The daily median time from message receipt to crisis specialist triage ranges from 8 to 13 min, compared to 9 h before the deployment of the system. We demonstrate that a NLP-based machine learning model can reliably identify potential crisis chat messages in a telehealth setting. Our system integrates into existing clinical workflows, suggesting that with appropriate training, humans can successfully leverage ML systems to facilitate triage of crisis messages.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.