ArticleNPJ digital medicine2025
Evaluating base and retrieval augmented LLMs with document or online support for evidence based neurology.
Article in NPJ digital medicine, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 23 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
23 citing papers in PubMed.
- Improving Reliability and Explainability of Medical Question Answering Through Atomic Fact-Checking in Retrieval-Augmented Large Language Models: Creation and Validation Study.Journal of medical Internet research · 2026Article
- Delivering high-quality multiple sclerosis care in Germany: challenges and regional models.Neurological research and practice · 2026Review
- Evaluation Methods for Inference-Time Retrieval-Augmented and Graph Retrieval-Augmented Large Language Models in Health Care: Scoping Review.Journal of medical Internet research · 2026Article
- Review
- Evaluating the First CE-Marked LLM-Based Chatbot for Evidence-Based Neurology.European journal of neurology · 2026Article
- Precision Grounding: augmenting large language models with evidence-based databases for trustworthy genetic variant summarization.International journal of medical informatics · 2026Article
- Exploring attitudes and acceptance of artificial intelligence in multiple sclerosis from the patient perspective.PLOS digital health · 2026Article
- Generative AI-Powered Virtual Assistant for Guideline-Directed Medical Therapy Optimization.JACC. Advances · 2026Article
- Benchmarking Large Language Models and Prompt Engineering Strategies in Microsatellite Instability Cancers: Evaluation Study.Journal of medical Internet research · 2026Article
- ONCO-RADS-guided Large Language Models for Extraction and Classification of Incidental Findings on Whole-Body Imaging Reports.Radiology. Imaging cancer · 2026Article
- Reframing AI for Rare Disease Recognition.Research square · 2026Article
- Disclaimers and Referral Patterns for Medical Advice Across Urgency Levels: Large Language Model Evaluation Study.Journal of medical Internet research · 2026Article
- MS360°: a conceptual digital-first, data-driven hybrid care framework for personalised multiple sclerosis management.NPJ digital medicine · 2026Review
- GEN-KnowRD: Reframing AI for Rare Disease Recognition.medRxiv : the preprint server for health sciences · 2026Article
- Development of retrieval-augmented generation-based large language model for drug-induced liver injury using Livertox data.Hepatology communications · 2026Article
- Evaluating Retrieval-Augmented Generation-Large Language Models for Infective Endocarditis Prophylaxis: Clinical Accuracy and Efficiency.International dental journal · 2026Article
- Evaluating reasoning large language models with human-like thinking in ophthalmic question answering.BMJ open ophthalmology · 2026Article
- A roadmap for medical large language models: a review of foundations, applications, and challenges.Military Medical Research · 2026Review
- Low-energy small language models with retrieval-augmented generation can surpass large-model performance in rheumatology.Frontiers in medicine · 2026Article
- Two-stage prompting framework with predefined verification steps for evaluating diagnostic reasoning tasks on two datasets.NPJ digital medicine · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
3 authors.
Funding
Abstract
Effectively managing evidence-based information is increasingly challenging. This study tested large language models (LLMs), including document- and online-enabled retrieval-augmented generation (RAG) systems, using 13 recent neurology guidelines across 130 questions. Results showed substantial variability. RAG improved accuracy compared to base models but still produced potentially harmful answers. RAG-based systems performed worse on case-based than knowledge-based questions. Further refinement and improved regulation is needed for safe clinical integration of RAG-enhanced LLMs.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.