ArticleJournal of substance abuse treatment2016
A Comparison of Natural Language Processing Methods for Automated Coding of Motivational Interviewing.
Article in Journal of substance abuse treatment, 2016. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 48 papers, 4 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
48 citing papers in PubMed, 4 syntheses or guidelines pooled it, 123 citations in OpenAlex.
- Natural language processing for mental health interventions: a systematic review and research framework.Translational psychiatry · 2023Pooled it
- Machine Learning and Natural Language Processing in Mental Health: Systematic Review.Journal of medical Internet research · 2021Pooled it
- Motivational interviewing quality assurance: A systematic review of assessment tools across research contexts.Clinical psychology review · 2020Pooled it
- A multivariate meta-analysis of motivational interviewing process and outcome.Psychology of addictive behaviors : journal of the Society of Psychologists in Addictive Behaviors · 2017Pooled it
- Improving Community-Based Care for Adolescents with ADHD: a Randomized Controlled Trial of Artificial Intelligence-Assisted Fidelity Supports.Prevention science : the official journal of the Society for Prevention Research · 2026Trial
- Capitalizing on natural language processing (NLP) to automate the evaluation of coach implementation fidelity in guided digital cognitive-behavioral therapy (GdCBT).Psychological medicine · 2025Trial
- Is fidelity to a complex behaviour change intervention associated with patient outcomes? Exploring the relationship between dietitian adherence and competence and the nutritional status of intervention patients in a successful stepped-wedge randomised clinical trial of eating as treatment (EAT).Implementation science : IS · 2021Trial
- Motivational interviewing training for community mental health providers: Findings and lessons learned from three related studies.PEC innovation · 2026Article
- Automating Motivational Interviewing Coding in Adolescent Substance Use Prevention: Human-AI Agreement Study.JMIR AI · 2026Article
- Natural language processing of patient in-session speech to predict brief motivational interviewing alcohol intervention response: an exploratory study.Alcohol and alcoholism (Oxford, Oxfordshire) · 2026Article
- Scoping review on natural language processing applications in counselling and psychotherapy.British journal of psychology (London, England : 1953) · 2026Article
- It Is the Journey, Not the Destination: Moving From End Points to Trajectories When Assessing Chatbot Mental Health Safety.JMIR mental health · 2026Article
- Positive psychology 4.0: a science of consequence for a world that can no longer wait.Frontiers in psychology · 2026Article
- Evaluation of an AI-Based Chatbot Providing Real-Time Feedback in Communication Training for Mental Health Care Professionals: Proof-of-Concept Observational Study.Journal of medical Internet research · 2025Observational
- Automating the assessment of multicultural orientation through machine learning and natural language processing.Psychotherapy (Chicago, Ill.) · 2025Article
- A Clinical Leadership Lens on Implementing Progress Feedback in Three Countries: Development of a Multidimensional Qualitative Coding Scheme.Administration and policy in mental health · 2025Article
- Developing an Implementation Model for ADHD Intervention in Community Clinics: Leveraging Artificial Intelligence and Digital Technology.Cognitive and behavioral practice · 2024Article
- Machine Learning-Based Evaluation of Suicide Risk Assessment in Crisis Counseling Calls.Psychiatric services (Washington, D.C.) · 2024Article
- Neural Conversational Agent for Weight Loss Counseling: Protocol for an Implementation and Feasibility Study.JMIR research protocols · 2024Article
- Automated Behavioral Coding to Enhance the Effectiveness of Motivational Interviewing in a Chat-Based Suicide Prevention Helpline: Secondary Analysis of a Clinical Trial.Journal of medical Internet research · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors at 2 institutions in 1 country.
Funding
Abstract
Motivational interviewing (MI) is an efficacious treatment for substance use disorders and other problem behaviors. Studies on MI fidelity and mechanisms of change typically use human raters to code therapy sessions, which requires considerable time, training, and financial costs. Natural language processing techniques have recently been utilized for coding MI sessions using machine learning techniques, rather than human coders, and preliminary results have suggested these methods hold promise. The current study extends this previous work by introducing two natural language processing models for automatically coding MI sessions via computer. The two models differ in the way they semantically represent session content, utilizing either 1) simple discrete sentence features (DSF model) and 2) more complex recursive neural networks (RNN model). Utterance- and session-level predictions from these models were compared to ratings provided by human coders using a large sample of MI sessions (N=341 sessions; 78,977 clinician and client talk turns) from 6 MI studies. Results show that the DSF model generally had slightly better performance compared to the RNN model. The DSF model had "good" or higher utterance-level agreement with human coders (Cohen's kappa>0.60) for open and closed questions, affirm, giving information, and follow/neutral (all therapist codes); considerably higher agreement was obtained for session-level indices, and many estimates were competitive with human-to-human agreement. However, there was poor agreement for client change talk, client sustain talk, and therapist MI-inconsistent behaviors. Natural language processing methods provide accurate representations of human derived behavioral codes and could offer substantial improvements to the efficiency and scale in which MI mechanisms of change research and fidelity monitoring are conducted.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.