ArticleJournal of medical Internet research2023
Evaluating Conversational Agents for Mental Health: Scoping Review of Outcomes and Outcome Measurement Instruments.
Article in Journal of medical Internet research, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 28 papers, 3 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
28 citing papers in PubMed, 3 syntheses or guidelines pooled it.
- Unleashing the potential of chatbots in mental health: bibliometric analysis.Frontiers in psychiatry · 2025Pooled it
- Self-Administered Interventions Based on Natural Language Processing Models for Reducing Depressive and Anxiety Symptoms: Systematic Review and Meta-Analysis.JMIR mental health · 2024Pooled it
- Attrition in Conversational Agent-Delivered Mental Health Interventions: Systematic Review and Meta-Analysis.Journal of medical Internet research · 2024Pooled it
- An Automated Conversational Agent Self-Help Program: Randomized Controlled Trial.Journal of medical Internet research · 2024Trial
- Body Image and Eating Disorder Education Chatbot, JEM, in Australia and Canada: First 6-Month Real-World Survey Evaluation.JMIR human factors · 2026Article
- Real-World Engagement With a Generative AI Conversational Agent for Mental Health Support: Retrospective Descriptive Study.JMIR formative research · 2026Article
- The Technologically Mediated Motherhood Constellation: Smartphone and AI as Novel Actors in Maternal Psychic Organization.JMIR pediatrics and parenting · 2026Article
- Safety Mechanisms and Risk Mitigation in Generative AI Mental Health Chatbots: A Systematic Scoping Review.Healthcare (Basel, Switzerland) · 2026Review
- It Is the Journey, Not the Destination: Moving From End Points to Trajectories When Assessing Chatbot Mental Health Safety.JMIR mental health · 2026Article
- Large language models for psychosocial risk assessment: A multi-method evaluation across suicide, intimate partner violence, and substance misuse.PLOS digital health · 2026Article
- Article
- Functional experiences of a LINE-based chatbot and their associations with system use experience among older adults: a cross-sectional study.Frontiers in public health · 2026Article
- User experience and safety of generative AI-based mental health chatbots: Scoping review protocol.PloS one · 2026Article
- Mental health chatbots and their technical features: A systematic review of reviews and a thematic analysis.Global mental health (Cambridge, England) · 2026Review
- Digital Psychiatry with Chatbot: Recent Advances and Limitations.Clinical psychopharmacology and neuroscience : the official scientific journal of the Korean College of Neuropsychopharmacology · 2025Review
- Toward a Smartphone-Based and Conversational Agent-Delivered Just-in-Time Adaptive Holistic Lifestyle Intervention for Older Adults Affected by Cognitive Decline: Two-Week Proof-of-Concept Study.JMIR formative research · 2025Article
- Evaluating the Quality of Psychotherapy Conversational Agents: Framework Development and Cross-Sectional Study.JMIR formative research · 2025Article
- Health Care Professionals' Engagement With Digital Mental Health Interventions in the United Kingdom and China: Mixed Methods Study on Engagement Factors and Design Implications.JMIR mental health · 2025Article
- Current Landscape and Future Directions for Mental Health Conversational Agents for Youth: Scoping Review.JMIR medical informatics · 2025Article
- Behavior Change Support Systems for Self-Treating Procrastination: Systematic Search in App Stores and Analysis of Motivational Design Archetypes.Journal of medical Internet research · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundRapid proliferation of mental health interventions delivered through conversational agents (CAs) calls for high-quality evidence to support their implementation and adoption. Selecting appropriate outcomes, instruments for measuring outcomes, and assessment methods are crucial for ensuring that interventions are evaluated effectively and with a high level of quality.
objectiveWe aimed to identify the types of outcomes, outcome measurement instruments, and assessment methods used to assess the clinical, user experience, and technical outcomes in studies that evaluated the effectiveness of CA interventions for mental health.
methodsWe undertook a scoping review of the relevant literature to review the types of outcomes, outcome measurement instruments, and assessment methods in studies that evaluated the effectiveness of CA interventions for mental health. We performed a comprehensive search of electronic databases, including PubMed, Cochrane Central Register of Controlled Trials, Embase (Ovid), PsychINFO, and Web of Science, as well as Google Scholar and Google. We included experimental studies evaluating CA mental health interventions. The screening and data extraction were performed independently by 2 review authors in parallel. Descriptive and thematic analyses of the findings were performed.
resultsWe included 32 studies that targeted the promotion of mental well-being (17/32, 53%) and the treatment and monitoring of mental health symptoms (21/32, 66%). The studies reported 203 outcome measurement instruments used to measure clinical outcomes (123/203, 60.6%), user experience outcomes (75/203, 36.9%), technical outcomes (2/203, 1.0%), and other outcomes (3/203, 1.5%). Most of the outcome measurement instruments were used in only 1 study (150/203, 73.9%) and were self-reported questionnaires (170/203, 83.7%), and most were delivered electronically via survey platforms (61/203, 30.0%). No validity evidence was cited for more than half of the outcome measurement instruments (107/203, 52.7%), which were largely created or adapted for the study in which they were used (95/107, 88.8%).
conclusionsThe diversity of outcomes and the choice of outcome measurement instruments employed in studies on CAs for mental health point to the need for an established minimum core outcome set and greater use of validated instruments. Future studies should also capitalize on the affordances made available by CAs and smartphones to streamline the evaluation and reduce participants' input burden inherent to self-reporting.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.