ArticleSurgical endoscopy2024
Dr. Google to Dr. ChatGPT: assessing the content and quality of artificial intelligence-generated medical information on appendicitis.
Article in Surgical endoscopy, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 47 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
47 citing papers in PubMed.
- Effects of an LLM-based intelligent adaptive teaching program on clinical risk perception among newly employed nurses: a quasi-experimental study.Frontiers in medicine · 2026Trial
- Performance of large language models in providing patient- oriented information on gingival recession: a comparative study.The Saudi dental journal · 2026Article
- An expert-based assessment of the quality, reliability, understandability, and readability of ChatGPT responses on cerebral palsy.Child's nervous system : ChNS : official journal of the International Society for Pediatric Neurosurgery · 2026Article
- Large Language Model Responses to Common Otolaryngological Questions: Evaluating Accuracy, Appropriateness, Readability, and Hallucinations.Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery · 2026Observational
- Agreement in Staging and Treatment Recommendations Among Clinicians, Text-Only Clinicians, DeepSeek-V3, and ChatGPT-4o for Nasopharyngeal Carcinoma Patients.Cancer medicine · 2026Article
- The potential of large language models to address patients' preoperative questions before anterior cervical discectomy and fusion surgery.Annals of translational medicine · 2026Article
- Evaluation of Large Language Model-Based Chatbots for Dental Trauma Management: A Comparative Study Based on Accuracy, Consistency and Information Quality.Australian endodontic journal : the journal of the Australian Society of Endodontology Inc · 2026Article
- Quality, completeness, readability, and response time of AI chatbots in the management of deep caries and pulp exposure: a bilingual comparative study.BMC oral health · 2026Article
- Can artificial intelligence applications be utilised in the management of dental trauma? A comparison of ChatGPT, Gemini, and DeepSeek.BMC oral health · 2026Article
- Comparing the Accuracy of ChatGPT-4o, DeepSeek-V3, and Gemini 2.5 Flash in Answering Frequently Asked Questions About Systemic Lupus Erythematosus: Quantitative Study.JMIR formative research · 2026Article
- Evaluating the Quality of Health Information: Comparison of Human and Artificial Intelligence.Neurogastroenterology and motility · 2026Article
- Evaluation of AI Chatbot Responses to a Standardized Patient Query on Myelin Oligodendrocyte Glycoprotein Antibody-Associated Disease: Cross-Sectional Content Analysis.JMIR medical informatics · 2026Article
- Patient and clinician engagement with generative artificial intelligence (GenAI): A scoping review of implications for patient-centered communication.Patient education and counseling · 2026Article
- AI-generated patient education for ankylosing spondylitis: a comparative study of readability and quality.Clinical rheumatology · 2026Article
- Evaluation of large Language models on pediatric asthma: a comparative study of Claude3-Opus, Gemini 2.0, ChatGPT-4o, and DeepSeek-a cross-sectional questionnaire study.BMC medical informatics and decision making · 2026Article
- Engaging Artificial Intelligence (AI)-based chatbots in digital health: A systematic review.PLOS digital health · 2026Article
- Can large language models provide quality, reliable and readable information about sudden infant death syndrome?BMC pediatrics · 2026Article
- ChatGPT-5.4 in health education: inter-generation stability and persistent readability challenges.Frontiers in public health · 2026Article
- Utility of large language models as information tools for nursing care in gout: a comparative study of DeepSeek and ChatGPT.Frontiers in medicine · 2026Article
- Re: Pushpanathan et al.: Can OpenAI's New o1 Model Outperform Its Predecessors in Common Eye Care Queries?Ophthalmology science · 2026Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
11 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
introductionGenerative artificial intelligence (AI) chatbots have recently been posited as potential sources of online medical information for patients making medical decisions. Existing online patient-oriented medical information has repeatedly been shown to be of variable quality and difficult readability. Therefore, we sought to evaluate the content and quality of AI-generated medical information on acute appendicitis.
methodsA modified DISCERN assessment tool, comprising 16 distinct criteria each scored on a 5-point Likert scale (score range 16-80), was used to assess AI-generated content. Readability was determined using the Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL) scores. Four popular chatbots, ChatGPT-3.5 and ChatGPT-4, Bard, and Claude-2, were prompted to generate medical information about appendicitis. Three investigators independently scored the generated texts blinded to the identity of the AI platforms.
resultsChatGPT-3.5, ChatGPT-4, Bard, and Claude-2 had overall mean (SD) quality scores of 60.7 (1.2), 62.0 (1.0), 62.3 (1.2), and 51.3 (2.3), respectively, on a scale of 16-80. Inter-rater reliability was 0.81, 0.75, 0.81, and 0.72, respectively, indicating substantial agreement. Claude-2 demonstrated a significantly lower mean quality score compared to ChatGPT-4 (p = 0.001), ChatGPT-3.5 (p = 0.005), and Bard (p = 0.001). Bard was the only AI platform that listed verifiable sources, while Claude-2 provided fabricated sources. All chatbots except for Claude-2 advised readers to consult a physician if experiencing symptoms. Regarding readability, FKGL and FRE scores of ChatGPT-3.5, ChatGPT-4, Bard, and Claude-2 were 14.6 and 23.8, 11.9 and 33.9, 8.6 and 52.8, 11.0 and 36.6, respectively, indicating difficulty readability at a college reading skill level.
conclusionAI-generated medical information on appendicitis scored favorably upon quality assessment, but most either fabricated sources or did not provide any altogether. Additionally, overall readability far exceeded recommended levels for the public. Generative AI platforms demonstrate measured potential for patient education and engagement about appendicitis.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.