ArticleOphthalmology and therapy2025
Assessment of Large Language Models in Cataract Care Information Provision: A Quantitative Comparison.
Article in Ophthalmology and therapy, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 14 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
14 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Evaluating Large Language Models in Ophthalmology: Systematic Review.Journal of medical Internet research · 2025Pooled it
- The doctors of the future: the competition of ChatGPT-4, ChatGPT-4 omni, and Gemini 2.0 Flash in andrology.BMC urology · 2026Article
- Application of large language models in clinical decision-making for dry eye disease.BMC medical informatics and decision making · 2026Article
- Assessing the Quality of AI Responses to Patient Concerns About Axial Spondyloarthritis: Delphi-Based Evaluation.JMIR AI · 2026Article
- Bilingual evaluation of large language models for patient education in refractive surgery.International journal of ophthalmology · 2026Article
- Assessing large language models as assistive tools in selecting first trial lens parameters for orthokeratology.Frontiers in medicine · 2026Article
- Evaluating the performance of large language models in sarcopenia-related patient queries: a foundational assessment for patient-centered validation.Frontiers in aging · 2026Article
- Comparison of Multiple State-of-the-Art Large Language Models for Patient Education Prior to CT and MRI Examinations.Journal of personalized medicine · 2025Article
- Evaluating the Efficacy of Large Language Models in Guiding Treatment Decisions for Pediatric Refractive Error.Ophthalmology and therapy · 2025Article
- Comparative assessment of artificial intelligence chatbots' performance in responding to healthcare professionals' and caregivers' questions about Dravet syndrome.Epilepsia open · 2025Article
- Evaluation of Three Large Language Models' Response Performances to Inquiries Regarding Post-Abortion Care in the Context of Chinese Language: A Comparative Analysis.Risk management and healthcare policy · 2025Article
- Large language models in the management of chronic ocular diseases: a scoping review.Frontiers in cell and developmental biology · 2025Review
- Quality of cataract-related videos on TikTok and its influencing factors: A cross-sectional study.Digital healthArticle
- Evaluating the reliability of large language models in answering FAQs for cataract surgery.Digital healthArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
Abstract
introductionCataracts are a significant cause of blindness. While individuals frequently turn to the Internet for medical advice, distinguishing reliable information can be challenging. Large language models (LLMs) have attracted attention for generating accurate, human-like responses that may be used for medical consultation. However, a comprehensive assessment of LLMs' accuracy within specific medical domains is still lacking.
methodsWe compiled 46 commonly inquired questions related to cataract care, categorized into six domains. Each question was presented to the LLMs, and three consultant-level ophthalmologists independently assessed the accuracy of their responses on a three-point scale (poor, borderline, good) and their comprehensiveness on a five-point scale. A majority consensus approach established the final rating for each response. Responses rated as 'Poor' were prompted for self-correction and reassessed.
resultsFor accuracy, ChatGPT-4o and Google Bard both achieved average sum scores of 8.7 (out of 9), followed by ChatGPT-3.5, Bing Chat, Llama 2, and Wenxin Yiyan. In consensus-based ratings, ChatGPT-4o outperformed Google Bard in the 'Good' rating. For completeness, ChatGPT-4o had the highest average sum score of 13.22 (out of 15), followed by Google Bard, ChatGPT-3.5, Llama 2, Bing Chat, and Wenxin Yiyan. Detailed performance data reveal nuanced differences in model capabilities. In the 'Prevention' domain, apart from Wenxin Yiyan, all other models were rated as 'Good'. All models showed improvement in self-correction. Bard and Bing improved 1/1 from 'Poor' to better, Llama improved 3/4, and Wenxin Yiyan improved 4/5.
conclusionsOur findings emphasize the potential of LLMs, particularly ChatGPT-4o, to deliver accurate and comprehensive responses to cataract-related queries, especially in prevention, indicating potential for medical consultation. Continuous efforts to enhance LLMs' accuracy through ongoing strategies and evaluations are essential.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.