ArticleJournal of medical Internet research2025
Ability of ChatGPT to Replace Doctors in Patient Education: Cross-Sectional Comparative Analysis of Inflammatory Bowel Disease.
Article in Journal of medical Internet research, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
16 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Impact of Large Language Model-Based AI Tools on Physician-Patient Communication: Systematic Review and Meta-Analysis.Journal of medical Internet research · 2026Pooled it
- Do bots provide correct and adequate guidance regarding acidity: A blinded comparison rated by patients and physicians.World journal of methodology · 2026Article
- Finite State Machine-Guided Retrieval-Augmented Generation Improves Expert-Rated Acceptability of a Peripherally Inserted Central Catheter Self-Management Chatbot: Single-Center Content Validation Study.Journal of medical Internet research · 2026Article
- Artificial intelligence in inflammatory bowel disease: bridging innovation, implementation and impact.Nature reviews. Gastroenterology & hepatology · 2026Review
- Evaluating Generative Artificial Intelligence Models' Responses to Questions About Scaphoid Fracture and Scaphoid Nonunion.Journal of hand surgery global online · 2026Article
- Feasibility of a multi-metric framework for evaluating patient-facing AI communication in cosmetic dentistry: an exploratory proof-of-concept study.Frontiers in oral health · 2026Article
- Generative AI-augmented transcriptomic and microbiome analysis across inflammatory and fibrotic disease states in Crohn's disease.Frontiers in artificial intelligence · 2026Article
- Construction and evaluation of the knowledge graph and large model question-answering system for Jin San Zhen therapy: a tool study for primary care and general practice.Frontiers in medicine · 2026Article
- Evaluation of AI-generated versus registered dietitian-authored nutrition responses: a cross-sectional study.mHealth · 2026Article
- The OB-GYN Take on GPT: Objective Assessment of Artificial Intelligence Models in Patient Education.Cureus · 2026Article
- Feeding intelligence: comparative evaluation of ChatGPT and clinical guidelines for nutritional management in head and neck cancer.Journal of translational medicine · 2025Article
- Large Language Model Synergy for Ensemble Learning in Medical Question Answering: Design and Evaluation Study.Journal of medical Internet research · 2025Article
- Assessing ChatGPT-v4 for Guideline-Concordant Inflammatory Bowel Disease: Accuracy, Completeness, and Temporal Drift.Journal of clinical medicine · 2025Article
- Large language models in clinical nutrition: an overview of its applications, capabilities, limitations, and potential future prospects.Frontiers in nutrition · 2025Review
- The reliability and quality of short videos as health information of guidance for bowel sounds: a cross-sectional study.Frontiers in public health · 2025Article
- Article
Corrections and comments
- Commented on by
Authors and funding
11 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundAlthough large language models (LLMs) such as ChatGPT show promise for providing specialized information, their quality requires further evaluation. This is especially true considering that these models are trained on internet text and the quality of health-related information available online varies widely.
objectiveThe aim of this study was to evaluate the performance of ChatGPT in the context of patient education for individuals with chronic diseases, comparing it with that of industry experts to elucidate its strengths and limitations.
methodsThis evaluation was conducted in September 2023 by analyzing the responses of ChatGPT and specialist doctors to questions posed by patients with inflammatory bowel disease (IBD). We compared their performance in terms of subjective accuracy, empathy, completeness, and overall quality, as well as readability to support objective analysis.
resultsIn a series of 1578 binary choice assessments, ChatGPT was preferred in 48.4% (95% CI 45.9%-50.9%) of instances. There were 12 instances where ChatGPT's responses were unanimously preferred by all evaluators, compared with 17 instances for specialist doctors. In terms of overall quality, there was no significant difference between the responses of ChatGPT (3.98, 95% CI 3.93-4.02) and those of specialist doctors (3.95, 95% CI 3.90-4.00; t
conclusionsChatGPT demonstrated more stable performance across various dimensions. Its output of health information content is more structurally sound, addressing the issue of variability in the information from individual specialist doctors. ChatGPT's performance highlights its potential as an auxiliary tool for health information, despite limitations such as artificial intelligence hallucinations. It is recommended that patients be involved in the creation and evaluation of health information to enhance the quality and relevance of the information.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.