ArticleCureus2025
Evaluating the Quality and Readability of Generative Artificial Intelligence (AI) Chatbot Responses in the Management of Achilles Tendon Rupture.
Article in Cureus, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 14 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
14 citing papers in PubMed.
- Flatfoot and artificial intelligence: Can language models be trusted for patient education?Medicine · 2026Article
- Comparative Expert Evaluation of Multimodal Large Language Models for Pediatric Rash Diagnosis: Clinical Utility, Safety, Information Quality, and Readability.Children (Basel, Switzerland) · 2026Article
- An expert-based assessment of the quality, reliability, understandability, and readability of ChatGPT responses on cerebral palsy.Child's nervous system : ChNS : official journal of the International Society for Pediatric Neurosurgery · 2026Article
- Evaluation of AI Chatbot Responses to a Standardized Patient Query on Myelin Oligodendrocyte Glycoprotein Antibody-Associated Disease: Cross-Sectional Content Analysis.JMIR medical informatics · 2026Article
- Investigating the Readability and Quality of AI Systems to Trending Questions About Food Poisoning.Journal of food science · 2026Article
- Performance of 5 AI Models on United States Medical Licensing Examination Step 1 Questions: Comparative Observational Study.JMIR AI · 2026Article
- Evaluating large language model responses to patient questions on ulnar collateral ligament repair.Clinics in shoulder and elbow · 2026Article
- Assessing the Utility of Large Language Models in Guiding Dental Practitioners on Pediatric Patient Care: A Comparative AI Study.Journal of clinical and experimental dentistry · 2025Article
- Artificial Intelligence in Orthopedic Surgery: Current Applications, Challenges, and Future Directions.MedComm · 2025Review
- A structured evaluation of LLM-generated step-by-step instructions in cadaveric brachial plexus dissection.BMC medical education · 2025Article
- Evaluating the Reliability and Quality of Sarcoidosis-Related Information Provided by AI Chatbots.Healthcare (Basel, Switzerland) · 2025Article
- Article
- Article
- ChatGPT as a source of surgical information: Evaluation of responses to patient questions on hallux rigidus fusion.Digital healthArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
9 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
introductionThe rise of artificial intelligence (AI), including generative chatbots like ChatGPT (OpenAI, San Francisco, CA, USA), has revolutionized many fields, including healthcare. Patients have gained the ability to prompt chatbots to generate purportedly accurate and individualized healthcare content. This study analyzed the readability and quality of answers to Achilles tendon rupture questions from six generative AI chatbots to evaluate and distinguish their potential as patient education resources.
methodsThe six AI models used were ChatGPT 3.5, ChatGPT 4, Gemini 1.0 (previously Bard; Google, Mountain View, CA, USA), Gemini 1.5 Pro, Claude (Anthropic, San Francisco, CA, USA) and Grok (xAI, Palo Alto, CA, USA) without prior prompting. Each was asked 10 common patient questions about Achilles tendon rupture, determined by five orthopaedic surgeons. The readability of generative responses was measured using Flesch-Kincaid Reading Grade Level, Gunning Fog, and SMOG (Simple Measure of Gobbledygook). The response quality was subsequently graded using the DISCERN criteria by five blinded orthopaedic surgeons.
resultsGemini 1.0 generated statistically significant differences in ease of readability (closest to average American reading level) than responses from ChatGPT 3.5, ChatGPT 4, and Claude. Additionally, mean DISCERN scores demonstrated significantly higher quality of responses from Gemini 1.0 (63.0±5.1) and ChatGPT 4 (63.8±6.2) than ChatGPT 3.5 (53.8±3.8), Claude (55.0±3.8), and Grok (54.2±4.8). However, the overall quality (question 16, DISCERN) of each model was averaged and graded at an above-average level (range, 3.4-4.4). DISCUSSION AND
conclusionOur results indicate that generative chatbots can potentially serve as patient education resources alongside physicians. Although some models lacked sufficient content, each performed above average in overall quality. With the lowest readability and highest DISCERN scores, Gemini 1.0 outperformed ChatGPT, Claude, and Grok and potentially emerged as the simplest and most reliable generative chatbot regarding management of Achilles tendon rupture.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.