ArticleAnnals of biomedical engineering2025
A Future of Self-Directed Patient Internet Research: Large Language Model-Based Tools Versus Standard Search Engines.
Article in Annals of biomedical engineering, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 7 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
7 citing papers in PubMed.
- Attitudes Toward Large Language Models in Health Care and Preferences for Their Adoption and Oversight Among Health Care Professionals: Cross-Sectional Survey.Journal of medical Internet research · 2026Article
- Large Language Model Performance and Clinical Reasoning Tasks.JAMA network open · 2026Article
- Preparing for Pandemics with Large Language Models: An Evaluation of Sensitivity Across COVID-19, Zika, and Monkeypox Case Reports.Journal of medical systems · 2026Article
- Comparing large language models and search engine responses to common orthodontic questions.PloS one · 2026Article
- The Digital Standardized Patient: An Artificial Intelligence Coach for Cultural Dexterity in Surgical Care.Journal of the American College of Surgeons · 2025Article
- Assessing the Capability of Large Language Models for Navigation of the Australian Health Care System: Comparative Study.JMIR AI · 2025Article
- Synthetic medical education in dermatology leveraging generative artificial intelligence.NPJ digital medicine · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
13 authors.
Funding
Abstract
purposeAs generalist large language models (LLMs) become more commonplace, patients will inevitably increasingly turn to these tools instead of traditional search engines. Here, we evaluate publicly available LLM-based chatbots as tools for patient education through physician review of responses provided by Google, Bard, GPT-3.5 and GPT-4 to commonly searched queries about prevalent chronic health conditions in the United States.
methodsFive distinct commonly Google-searched queries were selected for (i) hypertension, (ii) hyperlipidemia, (iii) diabetes, (iv) anxiety, and (v) mood disorders and prompted into each model of interest. Responses were assessed by board-certified physicians for accuracy, comprehensiveness, and overall quality on a five-point Likert scale. The Flesch-Kincaid Grade Levels were calculated to assess readability.
resultsGPT-3.5 (4.40 ± 0.48, 4.29 ± 0.43) and GPT-4 (4.35 ± 0.30, 4.24 ± 0.28) received higher ratings in comprehensiveness and quality than Bard (3.79 ± 0.36, 3.87 ± 0.32) and Google (1.87 ± 0.42, 2.11 ± 0.47), all p < 0.05. However, Bard (9.45 ± 1.35) and Google responses (9.92 ± 5.31) had a lower average Flesch-Kincaid Grade Level compared to GPT-3.5 (14.69 ± 1.57) and GPT-4 (12.88 ± 2.02), indicating greater readability.
conclusionThis study suggests that publicly available LLM-based tools may provide patients with more accurate responses to queries on chronic health conditions than answers provided by Google search. These results provide support for the use of these tools in place of traditional search engines for health-related queries.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.