ArticleScience advances2025
Benchmarking retrieval-augmented large language models in biomedical NLP: Application, robustness, and self-awareness.
Article in Science advances, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 12 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
12 citing papers in PubMed.
- Article
- Tutorial: guidance on the use of large language models for medical research.Nature protocols · 2026Review
- Efficient and Effective Internal Memory Retrieval for LLM-Based Healthcare Prediction.Findings of ACL. ACL · 2026Article
- Automated Identification of Nursing Diagnoses and Interventions From Nursing Records Using a Retrieval-Augmented Large Language Model Approach: Quantitative Study.Journal of medical Internet research · 2026Article
- MedQA-CS: Objective Structured Clinical Examination (OSCE)-Style Benchmark for Evaluating LLM Clinical Skills.Proceedings of the conference. Association for Computational Linguistics. European Chapter. Conference · 2026Article
- CancerLLM: a large language model in cancer domain.NPJ digital medicine · 2026Article
- A roadmap for medical large language models: a review of foundations, applications, and challenges.Military Medical Research · 2026Review
- MMRAG: multi-mode retrieval-augmented generation with large language models for biomedical in-context learning.Journal of the American Medical Informatics Association : JAMIA · 2025Article
- Large Language Model Synergy for Ensemble Learning in Medical Question Answering: Design and Evaluation Study.Journal of medical Internet research · 2025Article
- RAMIE: retrieval-augmented multi-task information extraction with large language models on dietary supplements.Journal of the American Medical Informatics Association : JAMIA · 2025Article
- BiomedRAG: A retrieval augmented large language model for biomedicine.Journal of biomedical informatics · 2025Article
- Improving electronic health record processing of large language models via retrieval-augmented generation: A case study on dietary supplements.AMIA ... Annual Symposium proceedings. AMIA Symposium · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
Abstract
To reduce hallucinations in large language models (LLMs), retrieval-augmented LLMs (RALs) retrieve supporting knowledge from external databases. However, their performance on biomedical natural language processing (NLP) tasks remains underexplored. We introduce Biomedical Retrieval-Augmented Generation Benchmark, a comprehensive evaluation framework assessing RALs across five biomedical NLP tasks and 11 datasets, using four testbeds: unlabeled robustness, counterfactual robustness, diverse robustness, and self-awareness. To improve RALs' robustness and negative awareness, we propose a detect-and-correct strategy and a contrastive learning approach. Experimental results show that RALs generally outperform standard LLMs on most biomedical tasks, but still struggle with robustness and self-awareness, particularly under counterfactual and diverse scenarios. Our proposed methods significantly improve performance in robustness to unlabeled and counterfactual data, and increase the model's ability to detect and avoid incorrect predictions. These findings highlight key limitations in current RALs and underscore the need for continued refinement to ensure reliability and accuracy in high-stakes biomedical applications.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.