ArticleJournal for immunotherapy of cancer2024
Use of artificial intelligence chatbots in clinical management of immune-related adverse events.
Article in Journal for immunotherapy of cancer, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 19 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
19 citing papers in PubMed.
- Large Language Models in Adverse Drug Reaction Detection and Pharmacovigilance: A Systematic Review of Current Applications, Challenges, and Future Directions.Diagnostics (Basel, Switzerland) · 2026Review
- Utility of Large Language Models for Congenital Microtia Reconstruction Education: Comparison of the Performance of Claude, GPT, and Gemini.Aesthetic plastic surgery · 2026Article
- Article
- Evaluating the Accuracy of Medical Information Generated by ChatGPT and Gemini and Its Alignment With International Clinical Guidelines From the Surviving Sepsis Campaign: Comparative Study.JMIR formative research · 2025Article
- Large Language Models for Drug-Related Adverse Events in Oncology Pharmacy: Detection, Grading, and Actioning.Pharmacy (Basel, Switzerland) · 2025Review
- Evaluation of the Quality and Reliability of ChatGPT-4's Responses on Allergen Immunotherapy Using Validated Instruments for Health Information Quality Assessment.Clinical and translational allergy · 2025Article
- From supportive care to strategic intervention: toward a multidisciplinary approach to cutaneous adverse events in cancer treatment.Supportive care in cancer : official journal of the Multinational Association of Supportive Care in Cancer · 2025Article
- Chatbot assistance in precision oncology treatment decision-making.The oncologist · 2025Article
- AI Chatbots in Oncology: A Comparative Study of Sider Fusion AI and Perplexity AI for Gastric Cancer Patients.Indian journal of surgical oncology · 2025Article
- Article
- Evaluating Large Language Models for Preoperative Patient Education in Superior Capsular Reconstruction: Comparative Study of Claude, GPT, and Gemini.JMIR perioperative medicine · 2025Article
- Clinical Management of Wasp Stings Using Large Language Models: Cross-Sectional Evaluation Study.Journal of medical Internet research · 2025Article
- Immune-related adverse events in immunotherapy: Challenges in diagnosis, monitoring, and management.Toxicology reports · 2025Article
- Optimizing Immunotherapy: The Synergy of Immune Checkpoint Inhibitors with Artificial Intelligence in Melanoma Treatment.Biomolecules · 2025Review
- Exploring the capabilities of GenAI for oral cancer consultations in remote consultations : Author.BMC oral health · 2025Article
- Large language models in oncology: a review.BMJ oncology · 2025Review
- Use of artificial intelligence chatbots in clinical management of immune-related adverse events.Journal for immunotherapy of cancer · 2024Article
- Understanding AI's Role in Endometriosis Patient Education and Evaluating Its Information and Accuracy: Systematic Review.JMIR AI · 2024Review
- Performance of Multimodal Artificial Intelligence Chatbots Evaluated on Clinical Oncology Cases.JAMA network open · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
12 authors.
Funding
Abstract
backgroundArtificial intelligence (AI) chatbots have become a major source of general and medical information, though their accuracy and completeness are still being assessed. Their utility to answer questions surrounding immune-related adverse events (irAEs), common and potentially dangerous toxicities from cancer immunotherapy, are not well defined.
methodsWe developed 50 distinct questions with answers in available guidelines surrounding 10 irAE categories and queried two AI chatbots (ChatGPT and Bard), along with an additional 20 patient-specific scenarios. Experts in irAE management scored answers for accuracy and completion using a Likert scale ranging from 1 (least accurate/complete) to 4 (most accurate/complete). Answers across categories and across engines were compared.
resultsOverall, both engines scored highly for accuracy (mean scores for ChatGPT and Bard were 3.87 vs 3.5, p<0.01) and completeness (3.83 vs 3.46, p<0.01). Scores of 1-2 (completely or mostly inaccurate or incomplete) were particularly rare for ChatGPT (6/800 answer-ratings, 0.75%). Of the 50 questions, all eight physician raters gave ChatGPT a rating of 4 (fully accurate or complete) for 22 questions (for accuracy) and 16 questions (for completeness). In the 20 patient scenarios, the average accuracy score was 3.725 (median 4) and the average completeness was 3.61 (median 4).
conclusionsAI chatbots provided largely accurate and complete information regarding irAEs, and wildly inaccurate information ("hallucinations") was uncommon. However, until accuracy and completeness increases further, appropriate guidelines remain the gold standard to follow.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.