Evidence map›Paper›PMID 41396219›Full record

ArticleWorld journal of urology2025

Systematic evaluation of deepseek in urolithiasis: from medical knowledge to clinical decision support.

Anguo Zhao, Rongkang Li, Lei Peng, Rui Liang, Ruonan Sun, Fan Wu, Zhengyan Wang, Xiaojian Xu, Jun Zhang, Jianquan Hou

Abstract readEvaluation Study
PubMed Publisher
In one paragraph

Article in World journal of urology, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.

0numbers the graph read from it
0cells of the map it votes in
0citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

0 citing papers in PubMed.

No citing paper in PubMed yet.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

10 authors.

Anguo ZhaoDepartment of Urology, The First Affiliated Hospital of Soochow University, Suzhou, 215000, China.
Rongkang LiDepartment of Urology, Lanzhou University Second Hospital, Lanzhou, 730000, People's Republic of China.
Lei PengDepartment of Urology, Lanzhou University Second Hospital, Lanzhou, 730000, People's Republic of China.
Rui LiangDepartment of Urology, The Fourth Affiliated Hospital of Soochow University, Medical Center of Soochow University, Suzhou Dushu Lake Hospital, Suzhou, 215000, China.
Ruonan SunWest China School of Medicine, Sichuan University, 610041, Chengdu, China.
Fan WuFaculty of Intelligent Manufacturing and Control Engineering, Shanghai Polytechnic University, Shanghai, 201209, China.
Zhengyan WangDepartment of Urology, Honghe Hospital Affiliated to Kunming Medical University/South Yunnan Central Hospital of Yunnan Province, Yunnan, 650500, Kunming, China.
Xiaojian XuDepartment of Urology, The First Affiliated Hospital of Soochow University, Suzhou, 215000, China. xxjsuda2011@163.com.
Jun ZhangDepartment of Urology, The First Affiliated Hospital of Soochow University, Suzhou, 215000, China. 15850115650@163.com.
Jianquan HouDepartment of Urology, The Fourth Affiliated Hospital of Soochow University, Medical Center of Soochow University, Suzhou Dushu Lake Hospital, Suzhou, 215000, China. houjianquan@suda.edu.cn.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundLarge language models (LLMs), such as ChatGPT, have demonstrated promising potential in medical knowledge retrieval and clinical decision support. DeepSeek, a China-developed model released in 2025, has been proposed as a medical AI tool, but its performance in healthcare settings remains underexplored.

methodsWe systematically evaluated DeepSeek's performance in urolithiasis through two approaches. First, we measured its accuracy and consistency using 157 single-choice questions from public datasets and the Chinese National Medical Licensing Examination. Second, we compared the clinical decision-making of DeepSeek and ChatGPT using three real-world urolithiasis cases. Responses were evaluated against those of clinicians across four dimensions: readability, medical knowledge accuracy, diagnostic appropriateness, and logical coherence.

resultsIn the medical knowledge task, DeepSeek achieved an accuracy above 83%, comparable to ChatGPT, with no significant difference in readability. However, in simulated clinical scenarios, DeepSeek underperformed in diagnostic reasoning and in avoiding unnecessary testing. The DeepSeek-R1 (R1) model scored significantly lower than both ChatGPT-o3 (R3) and physicians across several dimensions.

conclusionDeepSeek shows strong potential in structured medical knowledge retrieval but remains limited in its ability to support clinical decision-making. With continued model refinement, it may serve as a valuable tool in medical education and clinical practice.

Indexed as

Artificial IntelligenceClinical Decision-MakingDecision Support Systems, ClinicalUrolithiasisHumansChatGPTClinical decision supportDeepSeekLarge language modelsMedical educationUrolithiasis

Identifiers

PMID41396219

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.