ArticleJMIR mental health2024
A Comparison of ChatGPT and Fine-Tuned Open Pre-Trained Transformers (OPT) Against Widely Used Sentiment Analysis Tools: Sentiment Analysis of COVID-19 Survey Data.
Article in JMIR mental health, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 26 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
26 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Testing and Evaluation of Health Care Applications of Large Language Models: A Systematic Review.JAMA · 2025Pooled it
- Large Language Models for Mental Health Applications: Systematic Review.JMIR mental health · 2024Pooled it
- Large Language Models' Simulation of Patient Comprehension of Discharge Summaries: Multimodel Evaluation Study.Journal of medical Internet research · 2026Article
- Daily Language as an Objective Indicator of Depressive Affective States: A Two-Week Ecological Momentary Assessment Study in Emotional Labor Workers.Psychiatry investigation · 2026Article
- Global Sentiment Toward Health AI at the Dawn of the ChatGPT Era: Empirical Analysis of Twitter (X) Discourse.Journal of medical Internet research · 2026Article
- Comparison of Artificial Intelligence Tools With Human Coding for Sentiment, Topic, and Thematic Analysis Tasks of Public Health Datasets During the COVID-19 Pandemic in Australia: Case Study.Online journal of public health informatics · 2026Article
- Review
- Do language families matter? Evaluating LLMs for sentiment analysis through a hierarchical cross-lingual lens.Frontiers in artificial intelligence · 2026Article
- Large Language Models for Cardiovascular Disease, Cancer, and Mental Disorders: A Review of Systematic Reviews.Healthcare (Basel, Switzerland) · 2025Review
- Evaluation of Machine Learning Methods Developed for Prediction and Diagnosis of Pneumonia: A Systematic Review.Health science reports · 2025Review
- Zero-Shot Extraction of Seizure Outcomes from Clinical Notes Using Generative Pretrained Transformers.Journal of healthcare informatics research · 2025Article
- Efficient Detection of Stigmatizing Language in Electronic Health Records via In-Context Learning: Comparative Analysis and Validation Study.JMIR medical informatics · 2025Article
- Use of Large Language Models to Classify Epidemiological Characteristics in Synthetic and Real-World Social Media Posts About Conjunctivitis Outbreaks: Infodemiology Study.Journal of medical Internet research · 2025Article
- The Applications of Large Language Models in Mental Health: Scoping Review.Journal of medical Internet research · 2025Article
- Large Language Models' Accuracy in Emulating Human Experts' Evaluation of Public Sentiments about Heated Tobacco Products on Social Media: Evaluation Study.Journal of medical Internet research · 2025Article
- Evolving techniques in sentiment analysis: a comprehensive review.PeerJ. Computer science · 2025Article
- Article
- Use of ChatGPT to Explore Gender and Geographic Disparities in Scientific Peer Review.Journal of medical Internet research · 2024Article
- Large Language Models in Biomedical and Health Informatics: A Review with Bibliometric Analysis.Journal of healthcare informatics research · 2024Article
- Large Language Models Can Enable Inductive Thematic Analysis of a Social Media Corpus in a Single Prompt: Human Validation Study.JMIR infodemiology · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
8 authors.
Funding
Abstract
backgroundHealth care providers and health-related researchers face significant challenges when applying sentiment analysis tools to health-related free-text survey data. Most state-of-the-art applications were developed in domains such as social media, and their performance in the health care context remains relatively unknown. Moreover, existing studies indicate that these tools often lack accuracy and produce inconsistent results.
objectiveThis study aims to address the lack of comparative analysis on sentiment analysis tools applied to health-related free-text survey data in the context of COVID-19. The objective was to automatically predict sentence sentiment for 2 independent COVID-19 survey data sets from the National Institutes of Health and Stanford University.
methodsGold standard labels were created for a subset of each data set using a panel of human raters. We compared 8 state-of-the-art sentiment analysis tools on both data sets to evaluate variability and disagreement across tools. In addition, few-shot learning was explored by fine-tuning Open Pre-Trained Transformers (OPT; a large language model [LLM] with publicly available weights) using a small annotated subset and zero-shot learning using ChatGPT (an LLM without available weights).
resultsThe comparison of sentiment analysis tools revealed high variability and disagreement across the evaluated tools when applied to health-related survey data. OPT and ChatGPT demonstrated superior performance, outperforming all other sentiment analysis tools. Moreover, ChatGPT outperformed OPT, exhibited higher accuracy by 6% and higher F-measure by 4% to 7%.
conclusionsThis study demonstrates the effectiveness of LLMs, particularly the few-shot learning and zero-shot learning approaches, in the sentiment analysis of health-related survey data. These results have implications for saving human labor and improving efficiency in sentiment analysis tasks, contributing to advancements in the field of automated sentiment analysis.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.