ArticleJournal of medical Internet research2024
Using Large Language Models to Support Content Analysis: A Case Study of ChatGPT for Adverse Event Detection.
Article in Journal of medical Internet research, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 12 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
12 citing papers in PubMed, 21 citations in OpenAlex.
- A Large Language Model for Extracting Post-marketing Adverse Drug Events from Clinical Notes in the Electronic Health Record.Drug safety · 2026Article
- Characterizing public comments via Regulations.gov in response to proposed cannabis rescheduling in the United States.Addiction (Abingdon, England) · 2026Article
- Generative Artificial Intelligence for Qualitative Methods in Health Research: Rapid Review.Journal of medical Internet research · 2026Review
- The use and methodological reporting of large language models in qualitative research: a scoping review.BMC medical research methodology · 2026Article
- Research on ChatGPT generated text detection model based on phonetic feature extraction and semantic features.Scientific reports · 2026Article
- Deciphering AI-chatbots utilization in clinical practice, research and education among Chinese medical students in generative AI era: a cross-sectional study.BMC medical education · 2026Article
- Transformer-Based Topic Modeling: Characterizing Cannabis Product Adverse Experiences Self-Reported as Requiring Medical Attention on Reddit.Journal of medical Internet research · 2026Article
- A computationally efficient biomedical text processing framework for pharmacovigilance: integrating low-rank adaptation and interpretable AI for adverse drug reaction detection.Medical & biological engineering & computing · 2026Article
- LLMs outperform outsourced human coders on complex textual analysis.Scientific reports · 2025Article
- Large Language Models' Clinical Decision-Making on When to Perform a Kidney Biopsy: Comparative Study.Journal of medical Internet research · 2025Article
- Extracting Symptoms of Complex Conditions From Online Discourse (Subreddit to Symptomatology): Lexicon-Based Approach.JMIR medical informatics · 2025Article
- Pharmacovigilance in the Age of Legalized Cannabis: Using Social Media to Monitor Drug-Drug Interactions Between Immunosuppressants and Cannabis-Derived Products.Drug safety · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors at 2 institutions in 1 country.
Funding
Abstract
This study explores the potential of using large language models to assist content analysis by conducting a case study to identify adverse events (AEs) in social media posts. The case study compares ChatGPT's performance with human annotators' in detecting AEs associated with delta-8-tetrahydrocannabinol, a cannabis-derived product. Using the identical instructions given to human annotators, ChatGPT closely approximated human results, with a high degree of agreement noted: 94.4% (9436/10,000) for any AE detection (Fleiss κ=0.95) and 99.3% (9931/10,000) for serious AEs (κ=0.96). These findings suggest that ChatGPT has the potential to replicate human annotation accurately and efficiently. The study recognizes possible limitations, including concerns about the generalizability due to ChatGPT's training data, and prompts further research with different models, data sources, and content analysis tasks. The study highlights the promise of large language models for enhancing the efficiency of biomedical research.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.