ArticleHeliyon2024
Benchmarking four large language models' performance of addressing Chinese patients' inquiries about dry eye disease: A two-phase study.
Article in Heliyon, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 14 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
14 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Evaluating Large Language Models in Ophthalmology: Systematic Review.Journal of medical Internet research · 2025Pooled it
- Machine learning approaches for EGFR mutation status prediction in NSCLC: an updated systematic review.Frontiers in oncology · 2025Pooled it
- Evaluating the Performance of Large Language Models for Breast Cancer Patient Education: A Comparative Study.Journal of cancer education : the official journal of the American Association for Cancer Education · 2026Article
- Application of large language models in clinical decision-making for dry eye disease.BMC medical informatics and decision making · 2026Article
- Large Language Models in Ophthalmology: A Bibliographic Analysis.Turkish journal of ophthalmology · 2026Article
- Comparative performance of GPT-4, GPT-o3, GPT-5, Gemini-3-Flash, and DeepSeek-R1 in ophthalmology question answering.Frontiers in cell and developmental biology · 2026Article
- Bilingual evaluation of large language models for patient education in refractive surgery.International journal of ophthalmology · 2026Article
- Assessing ChatGPT's Educational Potential in Lung Cancer Radiotherapy From Clinician and Patient Perspectives: Content Quality and Readability Analysis.JMIR cancer · 2025Article
- Application of Large Language Models in Stroke Rehabilitation Health Education: 2-Phase Study.Journal of medical Internet research · 2025Article
- Assessing the adherence of large language models to clinical practice guidelines in Chinese medicine: a content analysis.Frontiers in pharmacology · 2025Article
- Large language models in the management of chronic ocular diseases: a scoping review.Frontiers in cell and developmental biology · 2025Review
- There are significant differences among artificial intelligence large language models when answering scientific questions.Frontiers in artificial intelligence · 2025Article
- Making Chatbots more human: deep reasoning large language models in ophthalmology.Frontiers in medicine · 2025Review
- Comparative performance of ChatGPT and DeepSeek in interpreting the 2025 ESICM guidelines on sepsis fluid therapy.Digital healthArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
14 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Purpose: To evaluate the performance of four large language models (LLMs)-GPT-4, PaLM 2, Qwen, and Baichuan 2-in generating responses to inquiries from Chinese patients about dry eye disease (DED). Design: Two-phase study, including a cross-sectional test in the first phase and a real-world clinical assessment in the second phase. Subjects: Eight board-certified ophthalmologists and 46 patients with DED. Methods: The chatbots' responses to Chinese patients' inquiries about DED were assessed by the evaluation. In the first phase, six senior ophthalmologists subjectively rated the chatbots' responses using a 5-point Likert scale across five domains: correctness, completeness, readability, helpfulness, and safety. Objective readability analysis was performed using a Chinese readability analysis platform. In the second phase, 46 representative patients with DED asked the two language models (GPT-4 and Baichuan 2) that performed best in the in the first phase questions and then rated the answers for satisfaction and readability. Two senior ophthalmologists then assessed the responses across the five domains. Main outcome measures: Subjective scores for the five domains and objective readability scores in the first phase. The patient satisfaction, readability scores, and subjective scores for the five-domains in the second phase. Results: In the first phase, GPT-4 exhibited superior performance across the five domains (correctness: 4.47; completeness: 4.39; readability: 4.47; helpfulness: 4.49; safety: 4.47, Conclusions: The findings underscore the potential of LLMs, particularly that of GPT-4 and Baichuan 2, in delivering accurate and comprehensive responses to questions from Chinese patients about DED.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.