Evidence map›Paper›PMID 38912297›Full record

ArticleProceedings of the SIGCHI conference on human factors in computing systems. CHI Conference2024

Towards AI-Driven Healthcare: Systematic Optimization, Linguistic Analysis, and Clinicians' Evaluation of Large Language Models for Smoking Cessation Interventions.

Paul Calle, Ruosi Shao, Yunlong Liu, Emily T Hébert, Darla Kendzor, Jordan Neil, Michael Businelle, Chongle Pan

Abstract read
In one paragraph

Article in Proceedings of the SIGCHI conference on human factors in computing systems. CHI Conference, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 8 papers.

0numbers the graph read from it
0cells of the map it votes in
8citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

8 citing papers in PubMed.

  1. Trial
  2. Trial
  3. Article
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

8 authors.

Paul CalleSchool of Computer Science, University of Oklahoma Norman, Oklahoma, USA.
Ruosi ShaoTSET Health Promotion Research Center, Stephenson Cancer Center, University of Oklahoma Health Sciences Center Oklahoma City, Oklahoma, USA.
Yunlong LiuSchool of Computer Science, University of Oklahoma Norman, Oklahoma, USA.
Emily T HébertSchool of Public Health, The University of Texas Health Science Center at Houston Austin, TX, USA.
Darla KendzorTSET Health Promotion Research Center, Stephenson Cancer Center; Department of Family and Preventive Medicine, University of Oklahoma Health Sciences Center Oklahoma City, Oklahoma, USA.
Jordan NeilTSET Health Promotion Research Center, Stephenson Cancer Center; Department of Family and Preventive Medicine, University of Oklahoma Health Sciences Center Oklahoma City, Oklahoma, USA.
Michael BusinelleTSET Health Promotion Research Center, Stephenson Cancer Center; Department of Family and Preventive Medicine, University of Oklahoma Health Sciences Center Oklahoma City, Oklahoma, USA.
Chongle PanSchool of Computer Science, University of Oklahoma Norman, Oklahoma, USA.

Funding

Tissue Pathology Shared ResourceP30CA225520 · NCI · UNIVERSITY OF OKLAHOMA HLTH SCIENCES CTR · PI Michael S. Businelle · 2018 to 2026
$27.1M
Smartphone Based Smoking Cessation Intervention for Socioeconomically Disadvantaged AdultsR01CA221819 · NCI · UNIVERSITY OF OKLAHOMA HLTH SCIENCES CTR · PI BUSINELLE, MICHAEL S. · 2019 to 2023
$3.0M
Using Machine Learning to Develop Just-in-Time Adaptive Interventions for Smoking CessationR00DA046564 · NIDA · UNIVERSITY OF TEXAS HLTH SCI CTR HOUSTON · PI HEBERT, EMILY TAYLOR · 2021 to 2023
$747k
NCI NIH HHS P30 CA225520NCI NIH HHS R01 CA221819NIDA NIH HHS R00 DA046564
6 · The paper itself

Abstract

Creating intervention messages for smoking cessation is a labor-intensive process. Advances in Large Language Models (LLMs) offer a promising alternative for automated message generation. Two critical questions remain: 1) How to optimize LLMs to mimic human expert writing, and 2) Do LLM-generated messages meet clinical standards? We systematically examined the message generation and evaluation processes through three studies investigating prompt engineering (Study 1), decoding optimization (Study 2), and expert review (Study 3). We employed computational linguistic analysis in LLM assessment and established a comprehensive evaluation framework, incorporating automated metrics, linguistic attributes, and expert evaluations. Certified tobacco treatment specialists assessed the quality, accuracy, credibility, and persuasiveness of LLM-generated messages, using expert-written messages as the benchmark. Results indicate that larger LLMs, including ChatGPT, OPT-13B, and OPT-30B, can effectively emulate expert writing to generate well-written, accurate, and persuasive messages, thereby demonstrating the capability of LLMs in augmenting clinical practices of smoking cessation interventions.

Indexed as

Computational Linguistic AnalysisExpert ReviewLarge Language ModelMessage GenerationSmoking Cessation Intervention

Identifiers

PMID38912297
PMCPMC11192205

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.