ArticleEye (London, England)2024
Google Gemini and Bard artificial intelligence chatbot performance in ophthalmology knowledge assessment.
Article in Eye (London, England), 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 26 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
26 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Evaluating Large Language Models in Ophthalmology: Systematic Review.Journal of medical Internet research · 2025Pooled it
- Exploratory study of large language models in surgical decision-making for lumbar disc herniation: a multicenter analysis based on multisource clinical information.BMC medical informatics and decision making · 2026Article
- A comprehensive review of intelligent question-answering systems in traditional Chinese medicine based on LLMs.Journal of pharmaceutical analysis · 2026Review
- Artificial intelligence in surgical care within low-income and middle-income countries: a scoping review of development, validation, and deployment.EClinicalMedicine · 2026Review
- Evaluation of the performance of four different large language models (ChatGPT, DeepSeek, Copilot, and Gemini) in answering oral, and maxillofacial radiology questions: pilot study.BMC oral health · 2026Article
- Evaluation of large language models in medical examinations: A scoping review protocol.PloS one · 2026Article
- Comparative performance of GPT-4, GPT-o3, GPT-5, Gemini-3-Flash, and DeepSeek-R1 in ophthalmology question answering.Frontiers in cell and developmental biology · 2026Article
- A comparative analysis of DeepSeek R1, DeepSeek-R1-Lite, OpenAi o1 Pro, and Grok 3 performance on ophthalmology board-style questions.Scientific reports · 2025Article
- Harnessing advanced large language models in otolaryngology board examinations: an investigation using python and application programming interfaces.European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery · 2025Article
- Diagnostic Performance of Publicly Available Large Language Models in Corneal Diseases: A Comparison with Human Specialists.Diagnostics (Basel, Switzerland) · 2025Article
- Effectiveness of various general large language models in clinical consensus and case analysis in dental implantology: a comparative study.BMC medical informatics and decision making · 2025Article
- Article
- Quality of Chatbot Responses to the Most Popular Questions Regarding Erectile Dysfunction.Urology research & practice · 2025Article
- Evaluation of large language model-generated medical information on idiopathic pulmonary fibrosis.Frontiers in artificial intelligence · 2025Article
- Enhancing ophthalmology students' awareness of retinitis pigmentosa: assessing the efficacy of ChatGPT in AI-assisted teaching of rare diseases-a quasi-experimental study.Frontiers in medicine · 2025Article
- Opportunities and Challenges of Chatbots in Ophthalmology: A Narrative Review.Journal of personalized medicine · 2024Review
- The future of AI clinicians: assessing the modern standard of chatbots and their approach to diagnostic uncertainty.BMC medical education · 2024Article
- Comparative performance analysis of large language models: ChatGPT-3.5, ChatGPT-4 and Google Gemini in glucocorticoid-induced osteoporosis.Journal of orthopaedic surgery and research · 2024Article
- Gemini-Assisted Deep Learning Classification Model for Automated Diagnosis of High-Resolution Esophageal Manometry Images.Medicina (Kaunas, Lithuania) · 2024Article
- Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
8 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
purposeWith the popularization of ChatGPT (Open AI, San Francisco, California, United States) in recent months, understanding the potential of artificial intelligence (AI) chatbots in a medical context is important. Our study aims to evaluate Google Gemini and Bard's (Google, Mountain View, California, United States) knowledge in ophthalmology.
methodsIn this study, we evaluated Google Gemini and Bard's performance on EyeQuiz, a platform containing ophthalmology board certification examination practice questions, when used from the United States (US). Accuracy, response length, response time, and provision of explanations were evaluated. Subspecialty-specific performance was noted. A secondary analysis was conducted using Bard from Vietnam, and Gemini from Vietnam, Brazil, and the Netherlands.
resultsOverall, Google Gemini and Bard both had accuracies of 71% across 150 text-based multiple-choice questions. The secondary analysis revealed an accuracy of 67% using Bard from Vietnam, with 32 questions (21%) answered differently than when using Bard from the US. Moreover, the Vietnam version of Gemini achieved an accuracy of 74%, with 23 (15%) answered differently than the US version of Gemini. While the Brazil (68%) and Netherlands (65%) versions of Gemini performed slightly worse than the US version, differences in performance across the various country-specific versions of Bard and Gemini were not statistically significant.
conclusionGoogle Gemini and Bard had an acceptable performance in responding to ophthalmology board examination practice questions. Subtle variability was noted in the performance of the chatbots across different countries. The chatbots also tended to provide a confident explanation even when providing an incorrect answer.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.