ArticleChildren (Basel, Switzerland)2024
Exploring the Role of ChatGPT-4, BingAI, and Gemini as Virtual Consultants to Educate Families about Retinopathy of Prematurity.
Article in Children (Basel, Switzerland), 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 20 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
20 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Evaluating Large Language Models in Ophthalmology: Systematic Review.Journal of medical Internet research · 2025Pooled it
- Evaluating large language models vs residents in cataract and refractive surgery: comparative analysis using the American Academy of Ophthalmology Self-Assessment Program.Journal of cataract and refractive surgery · 2026Trial
- Evaluation of AI Chatbot Responses to a Standardized Patient Query on Myelin Oligodendrocyte Glycoprotein Antibody-Associated Disease: Cross-Sectional Content Analysis.JMIR medical informatics · 2026Article
- Large Language Models in Ophthalmology: A Bibliographic Analysis.Turkish journal of ophthalmology · 2026Article
- Evolving Minds: Natural Learning vs. Artificial Learning in Ophthalmology Training.Turkish journal of ophthalmology · 2026Observational
- Accuracy and Reliability of AI Models in Emergency Myocardial Infarction Education.Emergency medicine international · 2026Article
- Comparison of different artificial intelligence tools' answers to questions related to early intervention: ChatGPT versus Gemini.Revista da Associacao Medica Brasileira (1992) · 2026Article
- Artificial Intelligence Models for Dysphonia Patient Education.Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery · 2025Article
- Evaluation of the readability, quality, and accuracy of AI chatbot responses to questions about deleterious oral habits.BMC oral health · 2025Article
- What Large Language Models offer about Familial Mediterranean Fever: An Analysis of Quality, Readability, Completeness, and Accuracy.Mediterranean journal of rheumatology · 2025Article
- Accuracy of ChatGPT, Gemini, Copilot, and Claude to Blepharoplasty-Related Questions.Aesthetic plastic surgery · 2025Article
- Development and evaluation of an agentic LLM based RAG framework for evidence-based patient education.BMJ health & care informatics · 2025Article
- ChatGPT-4o and OpenAI-o1: A Comparative Analysis of Its Accuracy in Refractive Surgery.Journal of clinical medicine · 2025Article
- Evaluating the Reliability and Quality of Sarcoidosis-Related Information Provided by AI Chatbots.Healthcare (Basel, Switzerland) · 2025Article
- Accuracy of LLMs in medical education: evidence from a concordance test with medical teacher.BMC medical education · 2025Article
- Evaluating the Use of Generative Artificial Intelligence to Support Genetic Counseling for Rare Diseases.Diagnostics (Basel, Switzerland) · 2025Article
- Evaluation of Responses to Questions About Keratoconus Using ChatGPT-4.0, Google Gemini and Microsoft Copilot: A Comparative Study of Large Language Models on Keratoconus.Eye & contact lens · 2025Article
- Opportunities and Challenges of Chatbots in Ophthalmology: A Narrative Review.Journal of personalized medicine · 2024Review
- Language discrepancies in the performance of generative artificial intelligence models: an examination of infectious disease queries in English and Arabic.BMC infectious diseases · 2024Article
- Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
3 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundLarge language models (LLMs) are becoming increasingly important as they are being used more frequently for providing medical information. Our aim is to evaluate the effectiveness of electronic artificial intelligence (AI) large language models (LLMs), such as ChatGPT-4, BingAI, and Gemini in responding to patient inquiries about retinopathy of prematurity (ROP).
methodsThe answers of LLMs for fifty real-life patient inquiries were assessed using a 5-point Likert scale by three ophthalmologists. The models' responses were also evaluated for reliability with the DISCERN instrument and the EQIP framework, and for readability using the Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (FKGL), and Coleman-Liau Index.
resultsChatGPT-4 outperformed BingAI and Gemini, scoring the highest with 5 points in 90% (45 out of 50) and achieving ratings of "agreed" or "strongly agreed" in 98% (49 out of 50) of responses. It led in accuracy and reliability with DISCERN and EQIP scores of 63 and 72.2, respectively. BingAI followed with scores of 53 and 61.1, while Gemini was noted for the best readability (FRE score of 39.1) but lower reliability scores. Statistically significant performance differences were observed particularly in the screening, diagnosis, and treatment categories.
conclusionChatGPT-4 excelled in providing detailed and reliable responses to ROP-related queries, although its texts were more complex. All models delivered generally accurate information as per DISCERN and EQIP assessments.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.