ArticleiGIE : innovation, investigation and insights2023
Evaluating the performance of ChatGPT in responding to questions about endoscopic procedures for patients.
Article in iGIE : innovation, investigation and insights, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 7 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
7 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Large Language Models for Chatbot Health Advice Studies: A Systematic Review.JAMA network open · 2025Pooled it
- Large Language Models in Gastroenterology: Systematic Review.Journal of medical Internet research · 2024Pooled it
- The potential utility of CHATGPT4.0 as an AI assistant in the education and management of patients with Barrett's esophagus.Diseases of the esophagus : official journal of the International Society for Diseases of the Esophagus · 2025Article
- Current applications and challenges in large language models for patient care: a systematic review.Communications medicine · 2025Article
- The Potential Clinical Utility of the Customized Large Language Model in Gastroenterology: A Pilot Study.Bioengineering (Basel, Switzerland) · 2024Article
- Exploring ChatGPT in clinical inquiry: a scoping review of characteristics, applications, challenges, and evaluation.Annals of medicine and surgery (2012) · 2024Article
- Digesting Digital Health: A Study of Appropriateness and Readability of ChatGPT-Generated Gastroenterological Information.Clinical and translational gastroenterology · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
10 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Background and Aims: We evaluated the precision, medical accuracy, superfluous content, and consistency of ChatGPT's responses to commonly asked questions about endoscopic procedures and its capability to provide emotional support, comparing its performance with the generative pretrained transformer 4 (GPT-4) model. Methods: A set of 113 questions related to EGD, colonoscopy, EUS, and ERCP was curated from professional societies and institutional web pages. Responses from ChatGPT were generated and subsequently graded by board-certified gastroenterologists and advanced endoscopists. The emotional support efficacy of ChatGPT and GPT-4 was also assessed by a board-certified psychiatrist (L.S.-M.). Results: ChatGPT exhibited moderate precision in answering questions about EGD (57.9% comprehensive), colonoscopy (47.6% comprehensive), EUS (48.1% comprehensive), and ERCP (44.4% comprehensive). Medical accuracy was highest for EGD (52.6% fully accurate) and lowest for EUS (40.7% fully accurate). Concerning superfluous content, responses were predominantly concise for EGD and colonoscopy, with ERCP and EUS showing increased extraneous content. Reproducibility scores varied across domains, ranging from 50.34% (for EUS) to 68.6% (for EGD). GPT-4 outperformed ChatGPT in emotional support, although both models exhibited satisfactory performance. Conclusions: ChatGPT delivers moderately precise and medically accurate answers related to common endoscopic procedures with varying levels of extraneous content. It holds promise as a supplementary information resource for both patients and healthcare professionals.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.