ArticleScientific reports2024
OpenMedLM: prompt engineering can out-perform fine-tuning in medical question-answering with open-source large language models.
Article in Scientific reports, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 31 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
31 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Performance and improvement strategies for adapting generative large language models for electronic health record applications: A systematic review.International journal of medical informatics · 2026Pooled it
- Generative large language models in medicine: a scoping review of recent methodological advances.npj health systems · 2026Review
- Stepwise Diagnostic Evaluation of Chinese Large Language Models: Comparative Study of Common and Rare Diseases.Journal of medical Internet research · 2026Article
- Generative AI, foundation models and large language models in radiation therapy physics: Clinical applications, challenges, and future directions.Medical physics · 2026Review
- A Large Language Model-Driven System for Advance Care Planning Training Among Health Care Providers in the Chinese Context: Development and Technical Evaluation.Journal of medical Internet research · 2026Article
- Susceptibility of Large Language Models to User-Driven Factors in Medical Queries.Journal of healthcare informatics research · 2026Article
- Article
- Enhancing quality of antimicrobial prescribing through 'Ask Eolas' (language model): a user-testing and simulation evaluation.npj antimicrobials and resistance · 2026Article
- Generative Artificial Intelligence Methodology Reporting in Otolaryngology: A Scoping Review.The Laryngoscope · 2026Article
- Scaling medical AI across clinical contexts.Nature medicine · 2026Review
- Performance comparison of large language models in boron neutron capture therapy knowledge assessment.Scientific reports · 2026Article
- Assessing data extraction in randomized clinical trials with large language models.BMC medical research methodology · 2026Article
- Structured clinical approach to enable large language models to be used for improved clinical diagnosis and explainable reasoning.Communications medicine · 2026Article
- Performance of large language models in delivering accurate and comprehensible patient information on heart failure and cardiomyopathy.Frontiers in digital health · 2026Article
- Application-driven pedagogical knowledge optimization of open-source LLMs via reinforcement learning and supervised fine-tuning.Frontiers in artificial intelligence · 2026Article
- Performance and safety of a fine-tuned small language model for pediatric emergency triage: A benchmark study.PloS one · 2026Article
- Exploring the potential of large language models for assessing medication adherence to the ESC heart failure guidelines.JAMIA open · 2025Article
- AI-Assisted Response Surface Methodology for Growth Optimization and Industrial Applicability Evaluation of the DiatomBioengineering (Basel, Switzerland) · 2025Article
- Leveraging ChatGPT and explainable AI for enhancing clinical decision support.Scientific reports · 2025Article
- Digital Cardiovascular Twins, AI Agents, and Sensor Data: A Narrative Review from System Architecture to Proactive Heart Health.Sensors (Basel, Switzerland) · 2025Review
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
10 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
LLMs can accomplish specialized medical knowledge tasks, however, equitable access is hindered by the extensive fine-tuning, specialized medical data requirement, and limited access to proprietary models. Open-source (OS) medical LLMs show performance improvements and provide the transparency and compliance required in healthcare. We present OpenMedLM, a prompting platform delivering state-of-the-art (SOTA) performance for OS LLMs on medical benchmarks. We evaluated OS foundation LLMs (7B-70B) on medical benchmarks (MedQA, MedMCQA, PubMedQA, MMLU medical-subset) and selected Yi34B for developing OpenMedLM. Prompting strategies included zero-shot, few-shot, chain-of-thought, and ensemble/self-consistency voting. OpenMedLM delivered OS SOTA results on three medical LLM benchmarks, surpassing previous best-performing OS models that leveraged costly and extensive fine-tuning. OpenMedLM displays the first results to date demonstrating the ability of OS foundation models to optimize performance, absent specialized fine-tuning. The model achieved 72.6% accuracy on MedQA, outperforming the previous SOTA by 2.4%, and 81.7% accuracy on MMLU medical-subset, establishing itself as the first OS LLM to surpass 80% accuracy on this benchmark. Our results highlight medical-specific emergent properties in OS LLMs not documented elsewhere to date and validate the ability of OS models to accomplish healthcare tasks, highlighting the benefits of prompt engineering to improve performance of accessible LLMs for medical applications.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.