ArticleNPJ digital medicine2024
Optimization of hepatological clinical guidelines interpretation by large language models: a retrieval augmented generation-based framework.
Article in NPJ digital medicine, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 94 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
94 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Generative large language models in the clinical management of Alzheimer's disease and mild cognitive impairment.Neurological sciences : official journal of the Italian Neurological Society and of the Italian Society of Clinical Neurophysiology · 2026Pooled it
- Improving large language model applications in biomedicine with retrieval-augmented generation: a systematic review, meta-analysis, and clinical development guidelines.Journal of the American Medical Informatics Association : JAMIA · 2025Pooled it
- Generative large language models in medicine: a scoping review of recent methodological advances.npj health systems · 2026Review
- Ensuring trustworthy AI assisted guideline development for clinical practice.NPJ digital medicine · 2026Article
- Evidence, use cases, and implementation safeguards of large language models in primary care.Communications medicine · 2026Review
- Article
- Role prompting modulates linguistic style but not clinical decision structure in GPT-5 tumour board simulation.NPJ digital medicine · 2026Article
- A Self-Controlled Benchmark of Retrieval-Augmented Generation for Large Language Models on Clinical Guideline Questions.Diagnostics (Basel, Switzerland) · 2026Article
- Evaluation Methods for Inference-Time Retrieval-Augmented and Graph Retrieval-Augmented Large Language Models in Health Care: Scoping Review.Journal of medical Internet research · 2026Article
- An automatic consult reply system for therapeutic plasma exchange using retrieval-augmented generation.Vox sanguinis · 2026Article
- Review
- Responsible Use of Large Language Models in Microbial Genomics and Bioinformatics: A Life-Science Framework for Reliability, Reproducibility, and Risk-Aware Interpretation.Life (Basel, Switzerland) · 2026Review
- Clinical large language model centered on electronic medical records.NPJ digital medicine · 2026Article
- Authoritative Textbook-Augmented Large Language Models for High-Altitude Public Health Medical Education in the Xizang Autonomous Region: Cross-Sectional Comparative Evaluation Study.Journal of medical Internet research · 2026Article
- Retrieval-Augmented Language Models for Clinical Decision Support in the Classification of Inborn Errors of Immunity.Journal of clinical immunology · 2026Article
- Use of Commercially Available Large Language Models to Generate Information Leaflets on Post-Intensive Care Syndrome: Clinical Utility Assessment.JMIR formative research · 2026Article
- Large Language Models for Diagnosis and Prognosis of Chronic Liver Diseases: A Systematic Review.Health science reports · 2026Review
- Automated Identification of Nursing Diagnoses and Interventions From Nursing Records Using a Retrieval-Augmented Large Language Model Approach: Quantitative Study.Journal of medical Internet research · 2026Article
- CuAgent provides a RAG-assisted intelligent framework to investigate cuproptosis.Scientific reports · 2026Article
- Leveraging Large Language Models for Accurate AO Fracture Classification from CT Text Reports.Journal of imaging informatics in medicine · 2026Article
34 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
Abstract
Large language models (LLMs) can potentially transform healthcare, particularly in providing the right information to the right provider at the right time in the hospital workflow. This study investigates the integration of LLMs into healthcare, specifically focusing on improving clinical decision support systems (CDSSs) through accurate interpretation of medical guidelines for chronic Hepatitis C Virus infection management. Utilizing OpenAI's GPT-4 Turbo model, we developed a customized LLM framework that incorporates retrieval augmented generation (RAG) and prompt engineering. Our framework involved guideline conversion into the best-structured format that can be efficiently processed by LLMs to provide the most accurate output. An ablation study was conducted to evaluate the impact of different formatting and learning strategies on the LLM's answer generation accuracy. The baseline GPT-4 Turbo model's performance was compared against five experimental setups with increasing levels of complexity: inclusion of in-context guidelines, guideline reformatting, and implementation of few-shot learning. Our primary outcome was the qualitative assessment of accuracy based on expert review, while secondary outcomes included the quantitative measurement of similarity of LLM-generated responses to expert-provided answers using text-similarity scores. The results showed a significant improvement in accuracy from 43 to 99% (p < 0.001), when guidelines were provided as context in a coherent corpus of text and non-text sources were converted into text. In addition, few-shot learning did not seem to improve overall accuracy. The study highlights that structured guideline reformatting and advanced prompt engineering (data quality vs. data quantity) can enhance the efficacy of LLM integrations to CDSSs for guideline delivery.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.