ArticleNature medicine2025
Benchmark evaluation of DeepSeek large language models in clinical decision-making.
Article in Nature medicine, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 132 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
132 citing papers in PubMed.
- Large language models enhance diagnostic reasoning of medical students in rheumatology: a randomized controlled trial.BMC medical education · 2026Trial
- ChatGPT Versus DeepSeek in Resectability Categorization of Chronic Rhinosinusitis: A Comparison Based on EPOS 2020.World journal of otorhinolaryngology - head and neck surgery · 2026Article
- Comparison of Two AI Chatbots for Diagnosis and Providing Treatment Suggestions in Retinopathy of Prematurity: Retrospective Study.Journal of medical Internet research · 2026Article
- Article
- Development of a Framework for Evaluating Large Language Model Safety and Reliability: a Proof-of-Concept Evaluation.Journal of medical systems · 2026Article
- Large Language Model Performance on Multistep Clinical Cases: Comparative Study Across Question and Case Levels.JMIR medical education · 2026Article
- Leveraging time-series electronic health records with large language models for chronic kidney disease diagnosis in primary care.BMJ health & care informatics · 2026Article
- LungGPT: A unified multimodal system for interpretable diagnosis and clinical decision support of respiratory diseases.Cell reports. Medicine · 2026Article
- A large language model-driven multidisciplinary AI agent system predicts delirium in emergency critically ill patients.Cell reports. Medicine · 2026Article
- Unmasking bias in the evidence ecosystem: a panoramic analysis of 311,751 meta-analyses using an artificial intelligence agent-based approach.Journal of global health · 2026Article
- A Secure, Scalable Large Language Model-Based System (CIDER) for High-Throughput Clinical Data Extraction From Medical Reports: Retrospective Validation Study.Journal of medical Internet research · 2026Article
- ASO Author Reflections: Benchmarking Large Language Models for Bladder Cancer: Comparative Knowledge Retrieval and Clinical Reasoning of DeepSeek and ChatGPT.Annals of surgical oncology · 2026Article
- Novelty to necessity: large language models in clinical practice-a call to action for physicians.Internal and emergency medicine · 2026Review
- Psychometric evaluation of the Spanish-language version of the health information technology usability evaluation scale (Health-ITUES).International journal of medical informatics · 2026Article
- Modern generative large language models in epilepsy care: a scoping review of current applications, challenges, and future directions.Acta epileptologica · 2026Review
- Error Detection and Correction in Chinese Radiology Reports Using Large Language Models: Real-World Clinical Validation Study.Journal of medical Internet research · 2026Article
- Critical Care-Specific vs General-Purpose Large Language Models in Emergency Intensive Care Unit Diagnosis: Single-Center Retrospective Paired Comparative Study.Journal of medical Internet research · 2026Article
- Development and Deployment of DeepSeek-Based Applications in Healthcare: A Chinese Perspective.Health care science · 2026Article
- Psychological Risk Assessment in Plastic Surgery via a DeepSeek Large Language Model: A Retrospective Cohort Study.Aesthetic plastic surgery · 2026Article
- Large reasoning models as thinking machines for medicine.Nature biomedical engineering · 2026Review
72 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
Abstract
Large language models (LLMs) are increasingly transforming medical applications. However, proprietary models such as GPT-4o face significant barriers to clinical adoption because they cannot be deployed on site within healthcare institutions, making them noncompliant with stringent privacy regulations. Recent advancements in open-source LLMs such as DeepSeek models offer a promising alternative because they allow efficient fine-tuning on local data in hospitals with advanced information technology infrastructure. Here, to demonstrate the clinical utility of DeepSeek-V3 and DeepSeek-R1, we benchmarked their performance on clinical decision support tasks against proprietary LLMs, including GPT-4o and Gemini-2.0 Flash Thinking Experimental. Using 125 patient cases with sufficient statistical power, covering a broad range of frequent and rare diseases, we found that DeepSeek models perform equally well and in some cases better than proprietary LLMs. Our study demonstrates that open-source LLMs can provide a scalable pathway for secure model training enabling real-world medical applications in accordance with data privacy and healthcare regulations.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.