SynthesisBMC medical informatics and decision making2025
Comparison of the readability of ChatGPT and Bard in medical communication: a meta-analysis.
Synthesis in BMC medical informatics and decision making, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 7 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
7 citing papers in PubMed, 1 synthesis or guideline pooled it.
- ChatGPT provides accurate and safe responses to patient questions on hip arthroscopy, while completeness remains variable: A systematic review and single-arm meta-analysis.Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA · 2026Pooled it
- The role of large language models in the writing of surgical reviews: fact or fantasy?Langenbeck's archives of surgery · 2026Article
- Assessing ChatGPT and Gemini Responses to Common Patient Questions Regarding Augmentation Mammaplasty.Aesthetic plastic surgery · 2026Article
- Article
- Evaluating GPT-4 Responses on Scars or Keloids for Patient Education: Large Language Model Evaluation Study.JMIR medical informatics · 2026Article
- Cross-platform evaluation of LLM-generated educational texts on cardiac myxoma: quality, readability, and actionability using network analysis and latent profile analysis.Frontiers in cardiovascular medicine · 2026Article
- The evaluation of tooth whitening from a perspective of artificial intelligence: a comparative analytical study.Frontiers in digital health · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundTo synthesize the results of various studies on the readability of ChatGPT and Bard in medical communication.
methodsSystemic literature research was conducted in PubMed, Ovid/Medline, CINAHL, Web-of-Science, Scopus and GoogleScholar to detect relevant publications (inclusion criteria: original research articles, English language, medical topic, ChatGPT-3.5/-4.0, Bard/Gemini, Flesch Reading Ease Score (FRE), Flesch Kincaid Grade Level (FKGL)). Study quality was analyzed using modified Downs-and-Black checklist (max. 8 points), adapted for studies on large language model. Analysis was performed on text simplification and/or text generation with ChatGPT-3.5/-4.0 versus Bard/Gemini. Meta-analysis was conducted, if outcome parameter was reported ≥ 3 studies. In addition, subgroup-analyses among different chatbot versions were performed. Publication bias was analyzed.
resultsOverall, 59 studies with 2342 items were analyzed. Study quality was limited with a mean of 6 points for FRE and 7 points for FKGL. Meta-analysis of text simplification for FRE between ChatGPT-3.5/-4.0 and Bard/Gemini was not significant (mean difference (MD):5.03; 95%-confidence interval (CI):-20.05,30.11; p = 0.48). FKGL of simplified texts of ChatGPT-3.5/-4.0 and Bard/Gemini was borderline significant (MD:-1.59; CI:-3.15,-0.04; p = 0.05) and subgroup-analysis between ChatGPT-4.0 and Bard was not significant (MD:-1.68; CI:-3.53,0.17; p = 0.07). Focused on text acquisition, MD for FRE and FKGL of studies on ChatGPT-3.5/-4.0- and Bard/Gemini-generated texts were significant (MD:-10.36; CI:-13.08,-7.64; p < 0.01 / MD:1.62; CI:1.09,2.15; p < 0.01). Subgroup-analysis of FRE was significant for ChatGPT-3.5 vs. Bard (MD:-16.07, CI:-24.90,-7.25; p < 0.01), ChatGPT-3.5 vs. Gemini (MD:-4.51; CI:-8.73,-0.29: p = 0.04), ChatGPT-4.0 vs. Bard (MD:-12.01, CI:-16.22,-7.81; p < 0.01) and ChatGPT-4.0 vs. Gemini (MD:-7.91, CI:-11.68,-4.15; p < 0.01). Analysis of FKGL in the subgroups was significant for ChatGPT-3.5 vs. Bard (MD:2.85, CI:1.98,3.73; p < 0.01), ChatGPT-3.5 vs. Gemini (MD:1.21, CI:0.50,1.93; p < 0.01) and ChatGPT-4.0 vs. Gemini (MD:1.95, CI:1.05,2.86; p < 0.01), but it was not significant for ChatGPT-4.0 vs. Bard (MD:0.64, CI:-0.46,1.74; p = 0.24). Egger's test was significant in text generation for FRE and FKGL (p < 0.01 / p < 0.01) and in subgroup ChatGPT-4.0 vs. Bard and ChatGPT-4.0 vs. Gemini (p < 0.01 / p = 0.02) for FRE as well as in subgroups ChatGPT-3.5 vs. Bard and ChatGPT-4.0 vs. Gemini for FKGL (p < 0.01 / p < 0.01).
conclusionReadability of spontaneously generated texts by Bard/Gemini was slightly superior compared to ChatGPT-3.5/-4.0 and readability of simplified texts by ChatGPT-3.5/-4.0 tended to be improved compared to Bard. Results are limited due study quality and publication bias. Standardized reporting could improve study quality and chatbot development.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.