ArticleJournal of medical Internet research2024
Enhancement of the Performance of Large Language Models in Diabetes Education through Retrieval-Augmented Generation: Comparative Study.
Article in Journal of medical Internet research, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 41 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
41 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Improving large language model applications in biomedicine with retrieval-augmented generation: a systematic review, meta-analysis, and clinical development guidelines.Journal of the American Medical Informatics Association : JAMIA · 2025Pooled it
- Trends in AI-based diagnosis and intervention of metabolic diseases: a bibliometric analysis of the literature from 2000 to 2024.Frontiers in medicine · 2025Pooled it
- A Conversational Agent for Providing Personalized Preexposure Prophylaxis (PrEP) Support: Protocol for Chatbot Implementation and Evaluation.JMIR research protocols · 2026Article
- Evaluating Guideline Adherence in Gemini-Powered Dental Trauma Workflows: Standalone Gemini Chat vs. Document-Grounded NotebookLM.Dental traumatology : official publication of International Association for Dental Traumatology · 2026Article
- Generative large language models in medicine: a scoping review of recent methodological advances.npj health systems · 2026Review
- Article
- Evaluation Methods for Inference-Time Retrieval-Augmented and Graph Retrieval-Augmented Large Language Models in Health Care: Scoping Review.Journal of medical Internet research · 2026Article
- An automatic consult reply system for therapeutic plasma exchange using retrieval-augmented generation.Vox sanguinis · 2026Article
- Assessing Quality, Readability, and Transparency of Online and AI-Generated Information on Type 2 Diabetes Mellitus: Cross-Sectional Exploratory Study.JMIR diabetes · 2026Article
- Authoritative Textbook-Augmented Large Language Models for High-Altitude Public Health Medical Education in the Xizang Autonomous Region: Cross-Sectional Comparative Evaluation Study.Journal of medical Internet research · 2026Article
- Augmenting large language models with clinical knowledge graph for personalized perioperative fluid therapy question answering.PLOS digital health · 2026Article
- Development and Evaluation of an Augmented Artificial Intelligence Model for Urogynecology Queries.International urogynecology journal · 2026Article
- Evaluating deepresearch and deepthink in total knee arthroplasty patient education: ChatGPT-4o excels in comprehensiveness, Deepseek R1 leads in clarity and readability of orthopedic information.Joint diseases and related surgery · 2026Article
- A multi-layer retrieval-augmented large language model framework for enhancing hypertension education.Hypertension research : official journal of the Japanese Society of Hypertension · 2026Article
- GastroTCM: a large language model assistant for gastroenterology in traditional Chinese medicine.Chinese medicine · 2026Article
- Creating customized chatbots with ChatGPT to promote physical activity: a mini review.Frontiers in digital health · 2026Review
- Performance comparison of large language models for medication counseling in people living with HIV.Frontiers in public health · 2026Article
- Evaluation of large language model-generated information in diabetes health patient education: a scoping review.Frontiers in public health · 2026Article
- Assessing the information quality of AI-generated patient educational materials for diabetes: a scoping review.Frontiers in public health · 2026Article
- From risk classification to clinical action: a prespecified paired pilot benchmark of public large language model interfaces for diabetes-related foot ulcer prevention.Frontiers in endocrinology · 2026Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
15 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundLarge language models (LLMs) demonstrated advanced performance in processing clinical information. However, commercially available LLMs lack specialized medical knowledge and remain susceptible to generating inaccurate information. Given the need for self-management in diabetes, patients commonly seek information online. We introduce the Retrieval-augmented Information System for Enhancement (RISE) framework and evaluate its performance in enhancing LLMs to provide accurate responses to diabetes-related inquiries.
objectiveThis study aimed to evaluate the potential of the RISE framework, an information retrieval and augmentation tool, to improve the LLM's performance to accurately and safely respond to diabetes-related inquiries.
methodsThe RISE, an innovative retrieval augmentation framework, comprises 4 steps: rewriting query, information retrieval, summarization, and execution. Using a set of 43 common diabetes-related questions, we evaluated 3 base LLMs (GPT-4, Anthropic Claude 2, Google Bard) and their RISE-enhanced versions respectively. Assessments were conducted by clinicians for accuracy and comprehensiveness and by patients for understandability.
resultsThe integration of RISE significantly improved the accuracy and comprehensiveness of responses from all 3 base LLMs. On average, the percentage of accurate responses increased by 12% (15/129) with RISE. Specifically, the rates of accurate responses increased by 7% (3/43) for GPT-4, 19% (8/43) for Claude 2, and 9% (4/43) for Google Bard. The framework also enhanced response comprehensiveness, with mean scores improving by 0.44 (SD 0.10). Understandability was also enhanced by 0.19 (SD 0.13) on average. Data collection was conducted from September 30, 2023 to February 5, 2024.
conclusionsThe RISE significantly improves LLMs' performance in responding to diabetes-related inquiries, enhancing accuracy, comprehensiveness, and understandability. These improvements have crucial implications for RISE's future role in patient education and chronic illness self-management, which contributes to relieving medical resource pressures and raising public awareness of medical knowledge.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.