ArticleThe British and Irish orthoptic journal2024
Comparative Analysis of Accuracy, Readability, Sentiment, and Actionability: Artificial Intelligence Chatbots (ChatGPT and Google Gemini) versus Traditional Patient Information Leaflets for Local Anesthesia in Eye Surgery.
Article in The British and Irish orthoptic journal, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 22 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
22 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Evaluating Large Language Models in Ophthalmology: Systematic Review.Journal of medical Internet research · 2025Pooled it
- Bridging the Health Literacy Gap for Patients With Oral Cancer: Readability Enhancement With AI Chatbots.Journal of oral pathology & medicine : official publication of the International Association of Oral Pathologists and the American Academy of Oral Pathology · 2026Article
- Caudal Anesthesia Patient Education Materials: Comprehensive Analysis of Generative Artificial Intelligence and Traditional Patient Information Leaflets.Paediatric anaesthesia · 2026Article
- Evaluation of large language model responses to patient questions on oral anticoagulant therapy: a comparative expert assessment.Exploratory research in clinical and social pharmacy · 2026Article
- LLMs in Medical Education for Autism Caregivers: A Comparative Evaluation of Accuracy, Readability, Actionability, and Neurodiversity-Affirming Language.Healthcare (Basel, Switzerland) · 2026Article
- Conversational Artificial Intelligence as a Source of Oral Health Information: A Cross-Sectional Study in a Romanian Population.Dentistry journal · 2026Article
- Article
- Accuracy, quality, and readability analyses of responses from large language models to questions on pediatric dental sedation.BMC oral health · 2026Article
- Quality, readability, and clinical-risk signals of default public-interface LLM responses to vascular and perioperative patient questions: a cross-sectional snapshot.Frontiers in public health · 2026Article
- Assessing large language model responses to pediatric depression FAQs: a cross-sectional study on readability, accuracy, and sentiment.Frontiers in psychiatry · 2026Article
- Comparative Evaluation of Popular Gen-AI Chatbots in Generating Patient Education Material on Pulmonary Artery Catheter Insertion.Annals of cardiac anaesthesia · 2026Article
- Evaluating the Accuracy of Medical Information Generated by ChatGPT and Gemini and Its Alignment With International Clinical Guidelines From the Surviving Sepsis Campaign: Comparative Study.JMIR formative research · 2025Article
- Feeding intelligence: comparative evaluation of ChatGPT and clinical guidelines for nutritional management in head and neck cancer.Journal of translational medicine · 2025Article
- Artificial intelligence in anesthesia: comparison of the utility of ChatGPT v/s google gemini large language models in pre-anesthetic education: content, readability and sentiment analysis.BMC anesthesiology · 2025Observational
- Evaluating the Response of AI-Based Large Language Models to Common Patient Concerns About Endodontic Root Canal Treatment: A Comparative Performance Analysis.Journal of clinical medicine · 2025Article
- Large language models as information providers for appropriate antimicrobial use: computational text analysis and expert-rated comparison of ChatGPT, Claude and Gemini.BMJ health & care informatics · 2025Article
- Article
- Readability of AI-Generated Patient Information Leaflets on Alzheimer's, Vascular Dementia, and Delirium.Cureus · 2025Article
- Patient education tools for chronic pain medications, AI versus traditional PILs - Clarifications.Indian journal of anaesthesia · 2024Article
- Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Background and Aim: Eye surgeries often evoke strong negative emotions in patients, including fear and anxiety. Patient education material plays a crucial role in informing and empowering individuals. Traditional sources of medical information may not effectively address individual patient concerns or cater to varying levels of understanding. This study aims to conduct a comparative analysis of the accuracy, completeness, readability, tone, and understandability of patient education material generated by AI chatbots versus traditional Patient Information Leaflets (PILs), focusing on local anesthesia in eye surgery. Methods: Expert reviewers evaluated responses generated by AI chatbots (ChatGPT and Google Gemini) and a traditional PIL (Royal College of Anaesthetists' PIL) based on accuracy, completeness, readability, sentiment, and understandability. Statistical analyses, including ANOVA and Tukey HSD tests, were conducted to compare the performance of the sources. Results: Readability analysis showed variations in complexity among the sources, with AI chatbots offering simplified language and PILs maintaining better overall readability and accessibility. Sentiment analysis revealed differences in emotional tone, with Google Gemini exhibiting the most positive sentiment. AI chatbots demonstrated superior understandability and actionability, while PILs excelled in completeness. Overall, ChatGPT showed slightly higher accuracy (scores expressed as mean ± standard deviation) (4.71 ± 0.5 vs 4.61 ± 0.62) and completeness (4.55 ± 0.58 vs 4.47 ± 0.58) compared to Google Gemini, but PILs performed best (4.84 ± 0.37 vs 4.88 ± 0.33) in terms of both accuracy and completeness (p-value for completeness <0.05). Conclusion: AI chatbots show promise as innovative tools for patient education, complementing traditional PILs. By leveraging the strengths of both AI-driven technologies and human expertise, healthcare providers can enhance patient education and empower individuals to make informed decisions about their health and medical care.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.