ArticleEye (London, England)2024
Assessing large language models' accuracy in providing patient support for choroidal melanoma.
Article in Eye (London, England), 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 19 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
19 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Evaluating Large Language Models in Ophthalmology: Systematic Review.Journal of medical Internet research · 2025Pooled it
- Accuracy of Large Language Models When Answering Clinical Research Questions: Systematic Review and Network Meta-Analysis.Journal of medical Internet research · 2025Pooled it
- Evaluating large language models vs residents in cataract and refractive surgery: comparative analysis using the American Academy of Ophthalmology Self-Assessment Program.Journal of cataract and refractive surgery · 2026Trial
- Accuracy and Effectiveness of AI-Powered Systems in Patient Counseling, Education, and Management in Optometry and Related Eye-Care Settings: A Systematic Review.Healthcare (Basel, Switzerland) · 2026Review
- Large Language Models in Ophthalmology: A Bibliographic Analysis.Turkish journal of ophthalmology · 2026Article
- Generative AI for patient education in cancer care: A scoping review of evaluation practices and emerging trends.Technical innovations & patient support in radiation oncology · 2026Review
- How Far Have Large Language Models Advanced in Ophthalmology? A Systematic Review of Their Development, Evaluation, and Readiness for Clinical Use.Research square · 2026Article
- Accuracy and Readability of Chat Generative Pre-Trained Transformer-4 Omni in Answering Ophthalmology Patient Questions.Ophthalmology science · 2026Article
- Assessing GPT-4o in cataract surgery decision-making: appropriateness, consistency, and clinical implications.Frontiers in artificial intelligence · 2026Article
- Artificial Intelligence in Ocular Oncology: Differentiating Choroidal Melanocytic Lesions.Ophthalmology science · 2026Article
- Evaluation of large language models in patient education for hyperthyroidism: A comparative study of chatgpt, gemini, and deepseek.Endocrine · 2025Article
- Global perspectives of ophthalmologists on artificial intelligence adoption in clinical practice.International journal of retina and vitreous · 2025Article
- [Application of large language models in health education for patients with pediatric cataract].Zhong nan da xue xue bao. Yi xue ban = Journal of Central South University. Medical sciences · 2025Article
- ChatGPT-4o and OpenAI-o1: A Comparative Analysis of Its Accuracy in Refractive Surgery.Journal of clinical medicine · 2025Article
- The Role of ChatGPT in Dermatology Diagnostics.Diagnostics (Basel, Switzerland) · 2025Review
- Opportunities and Challenges of Chatbots in Ophthalmology: A Narrative Review.Journal of personalized medicine · 2024Review
- Exploring the Role of Large Language Models in Melanoma: A Systematic Review.Journal of clinical medicine · 2024Review
- Artificial intelligence derived large language model in decision-making process in uveitis.International journal of retina and vitreous · 2024Article
- Performance analysis of an emergency triage system in ophthalmology using a customized CHATBOT.Digital healthArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
purposeThis study aimed to evaluate the accuracy of information that patients can obtain from large language models (LLMs) when seeking answers to common questions about choroidal melanoma.
methodsComparative study comparing frequently asked questions from choroidal melanoma patients and queried three major LLMs-ChatGPT 3.5, Bing AI, and DocsGPT. Answers were reviewed by three ocular oncology experts and scored as accurate, partially accurate, or inaccurate. Statistical analysis compared the quality of responses across models.
resultsFor medical advice questions, ChatGPT gave 92% accurate responses compared to 58% for Bing AI and DocsGPT. For pre/post-op questions, ChatGPT and Bing AI were 86% accurate while DocsGPT was 73% accurate. There were no statistically significant differences between models. ChatGPT responses were the longest while Bing AI responses were the shortest, but length did not affect accuracy. All LLMs appropriately directed patients to seek medical advice from professionals.
conclusionLLMs show promising capability to address common choroidal melanoma patient questions at generally acceptable accuracy levels. However, inconsistent, and inaccurate responses do occur, highlighting the need for improved fine-tuning and oversight before integration into clinical practice.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.