ArticleNature medicine2025
Comparative benchmarking of the DeepSeek large language model on medical tasks and clinical reasoning.
Article in Nature medicine, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 115 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
115 citing papers in PubMed.
- An LLM-enhanced AI agent for rapid diagnosis of acute aortic dissection in multi-center settings.iScience · 2026Article
- Performance of large language models in the Chinese National Medical Licensing Examinations and Beyond: Scoping review.PLOS digital health · 2026Article
- The 5R's of large language model-assisted diagnosis: A practical framework for hospitalists.Journal of hospital medicine · 2026Article
- Development of a Framework for Evaluating Large Language Model Safety and Reliability: a Proof-of-Concept Evaluation.Journal of medical systems · 2026Article
- [Application effect of LLMs as decision-support tools in emergency burn treatment].Zhonghua shao shang yu chuang mian xiu fu za zhi · 2026Article
- A large language model-driven multidisciplinary AI agent system predicts delirium in emergency critically ill patients.Cell reports. Medicine · 2026Article
- LungGPT: A unified multimodal system for interpretable diagnosis and clinical decision support of respiratory diseases.Cell reports. Medicine · 2026Article
- Large Language Model Simplification of Open Access Pediatric Strabismus Literature: Cross-Sectional Validation of Readability and Clinical Fidelity.JMIR formative research · 2026Article
- Psychological Risk Assessment in Plastic Surgery via a DeepSeek Large Language Model: A Retrospective Cohort Study.Aesthetic plastic surgery · 2026Article
- Large reasoning models as thinking machines for medicine.Nature biomedical engineering · 2026Review
- The Performance of ChatGPT-4o and DeepSeek-R1 in Interpreting Thyroid Nodule Ultrasound Text Reports: Multicenter Study.Journal of medical Internet research · 2026Article
- Dynamic Closed-Loop Medical Record Quality Management Using an AI-Driven Multilevel Quality Control System: Development and Implementation Study.JMIR medical informatics · 2026Article
- A biomimetic cryoprotectant preserves extracellular vesicles bioactivity for intravenous administration and potentiates wound healing.Journal of nanobiotechnology · 2026Article
- Value of the DeepSeek-R1 large language model in extracting structured data from magnetic resonance imaging reports of rectal cancer and assisting in tumor staging.Quantitative imaging in medicine and surgery · 2026Article
- Review
- WiNGPT-32B: An Open-Source, Locally Deployable LLM for RECIST Assessment via Chained Task Execution Using Radiology Report Text.Diagnostics (Basel, Switzerland) · 2026Article
- Association between sources of nutrition information and self-perceived nutrition competence of healthcare professionals: a cross-sectional study.BMC medical education · 2026Article
- Article
- From breadth to depth in clinical artificial intelligence evaluation.Nature biomedical engineering · 2026Article
- BRIDGE: benchmarking large language models for understanding real-world clinical practice texts.Nature biomedical engineering · 2026Article
55 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
23 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
DeepSeek is a newly introduced large language model (LLM) designed for enhanced reasoning, but its medical-domain capabilities have not yet been evaluated. Here we assessed the capabilities of three LLMs- DeepSeek-R1, ChatGPT-o1 and Llama 3.1-405B-in performing four different medical tasks: answering questions from the United States Medical Licensing Examination (USMLE), interpreting and reasoning on the basis of text-based diagnostic and management cases, providing tumor classification according to RECIST 1.1 criteria and providing summaries of diagnostic imaging reports across multiple modalities. In the USMLE test, the performance of DeepSeek-R1 (accuracy 0.92) was slightly inferior to that of ChatGPT-o1 (accuracy 0.95; P = 0.04) but better than that of Llama 3.1-405B (accuracy 0.83; P < 10
Indexed as
Identifiers
40267969What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.