ArticleNPJ digital medicine2025
Retrieval augmented generation for 10 large language models and its generalizability in assessing medical fitness.
Article in NPJ digital medicine, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT06949462 (Evaluating the Effectiveness of Large Language Models in Anaesthesia and Procedural Consent), which is not on this map. Cited by 63 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Evaluating the Effectiveness of Large Language Models in Anaesthesia and Procedural Consent: A Comparative Analysis With Traditional Patient Consent Methods
Who cites it
63 citing papers in PubMed.
- Concordance between GPT-4 and a multidisciplinary tumor board in pancreatic cancer: A prospective pilot study.Langenbeck's archives of surgery · 2026Article
- Local large language model-guided optimization for prostate radiotherapy planning.Medical physics · 2026Article
- The economics of accuracy for medical reasoning with large language models.PLOS digital health · 2026Article
- GPT-4o-Powered Preanesthetic AI: Development and Validation Study.JMIR medical informatics · 2026Article
- A Multidisciplinary Team-Based Large Language Model Framework for Predicting Postoperative Neurological Complications in Acute Type A Aortic Dissection: Model Development and Validation Study.Journal of medical Internet research · 2026Article
- An autonomous multimodal AI agent for evidence-grounded ophthalmic diagnosis.Cell reports. Medicine · 2026Article
- Intelligent Framework for Adverse Drug Event Identification Using Large Language Models and Retrieval-Augmented Generation: Development and Evaluation Study.Journal of medical Internet research · 2026Article
- Evaluation Methods for Inference-Time Retrieval-Augmented and Graph Retrieval-Augmented Large Language Models in Health Care: Scoping Review.Journal of medical Internet research · 2026Article
- ChatNSG: An Overview of Contemporary and Emerging Artificial Intelligence Models for the Neurosurgeon.Journal of neurological surgery. Part B, Skull base · 2026Article
- An automatic consult reply system for therapeutic plasma exchange using retrieval-augmented generation.Vox sanguinis · 2026Article
- Prediction of Postoperative Vomiting Within 24 Hours Using Machine Learning With Large Language Model-Enhanced Interpretability: Development and Validation Study.JMIR medical informatics · 2026Article
- A S.C.O.R.E. framework for evaluating open-ended responses from large language models in healthcare.Cell reports. Medicine · 2026Article
- Fine-Tuning, Retrieval-Augmented Generation, and Hybrid Large Language Models for Postoperative Decision Support: Comparative Analysis.Journal of medical Internet research · 2026Article
- A Controlled Comparison of Human and AI-Assisted Automated Revision of Delphi Statements on RNA-Based Medicines: Parallel, 2-Arm Study.JMIR medical informatics · 2026Article
- Artificial Intelligence Tools in Pre-Travel Health Consultations: A Scoping Review of Clinical Evidence, Implementation Gaps, and Emerging Opportunities.Tropical medicine and infectious disease · 2026Review
- Authoritative Textbook-Augmented Large Language Models for High-Altitude Public Health Medical Education in the Xizang Autonomous Region: Cross-Sectional Comparative Evaluation Study.Journal of medical Internet research · 2026Article
- EyeRAG: graph retrieval-augmented generation for safe and accurate clinical dialogue in ophthalmology.NPJ digital medicine · 2026Article
- Retrieval-Augmented Language Models for Clinical Decision Support in the Classification of Inborn Errors of Immunity.Journal of clinical immunology · 2026Article
- Augmenting large language models with clinical knowledge graph for personalized perioperative fluid therapy question answering.PLOS digital health · 2026Article
- A Bilingual AI-Based Chatbot for Nutrition Education in a Food Is Medicine Intervention for High-Risk Pregnant Women: Design and Development Study.JMIR formative research · 2026Article
3 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
13 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Large Language Models (LLMs) hold promise for medical applications but often lack domain-specific expertise. Retrieval Augmented Generation (RAG) enables customization by integrating specialized knowledge. This study assessed the accuracy, consistency, and safety of LLM-RAG models in determining surgical fitness and delivering preoperative instructions using 35 local and 23 international guidelines. Ten LLMs (e.g., GPT3.5, GPT4, GPT4o, Gemini, Llama2, and Llama3, Claude) were tested across 14 clinical scenarios. A total of 3234 responses were generated and compared to 448 human-generated answers. The GPT4 LLM-RAG model with international guidelines generated answers within 20 s and achieved the highest accuracy, which was significantly better than human-generated responses (96.4% vs. 86.6%, p = 0.016). Additionally, the model exhibited an absence of hallucinations and produced more consistent output than humans. This study underscores the potential of GPT-4-based LLM-RAG models to deliver highly accurate, efficient, and consistent preoperative assessments.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.