ArticleJournal of medical systems2025
Performance of DeepSeek-R1 and ChatGPT-4o on the Chinese National Medical Licensing Examination: A Comparative Study.
Article in Journal of medical systems, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 34 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
34 citing papers in PubMed.
- Diagnostic performance of large language models for discrepancy detection in non-English resident-authored abdominal CT and MRI reports: a Turkish-language evaluation.Abdominal radiology (New York) · 2026Article
- Large Language Model Performance on Multistep Clinical Cases: Comparative Study Across Question and Case Levels.JMIR medical education · 2026Article
- Large language models for analyzing contrast-enhanced ultrasound reports of pancreatic cystic lesions.Quantitative imaging in medicine and surgery · 2026Article
- Capacity of Understanding the Future Approaches in Cancer Treatment by Multiple Models of Artificial Intelligence.Journal of cancer education : the official journal of the American Association for Cancer Education · 2026Article
- Performance of DeepSeek-R1 and ChatGPT-5 in the Generation of North American Spine Society Clinical Guidelines for Adult Vertebral Compression Fractures: Comparative Study.Journal of medical Internet research · 2026Article
- Article
- Quality Evaluation of Large Language Model-Assisted Generation of Initial Senior Physician Ward Round Records for Patients With Acute Poisoning: Cross-Sectional Study.Journal of medical Internet research · 2026Article
- Evaluation of ChatGPT-4o's and DeepSeek R1's responses to urological problems: a comparative study.BMC urology · 2026Article
- Applications of DeepSeek in Medicine: Bibliometric Analysis and Scoping Review.Journal of medical Internet research · 2026Article
- How do large language models answer ADHD-related questions? A comparative study of ChatGPT, Gemini, and DeepSeek.BMC psychiatry · 2026Article
- Three large language models demonstrate competitive performance in Traditional Chinese Medicine national medical licensing examinations over two years.Scientific reports · 2026Article
- Comparative performance of LLMs and machine learning in predicting complications after percutaneous kyphoplasty for osteoporotic vertebral compression fractures.NPJ digital medicine · 2026Article
- Performance of five large language models in oral and maxillofacial surgery exam questions: a comparative study.BMC oral health · 2026Article
- Textbook-level medical knowledge in large language models: comparative evaluation using Japanese National Medical Examination.BMC medical informatics and decision making · 2026Article
- Evaluating the Clinical Competence of Large Language Models in Prostate Cancer Management: A Comparative Study of DeepSeek-R1 and ChatGPT.Annals of surgical oncology · 2026Article
- Evaluating the performance of large language models in sarcopenia-related patient queries: a foundational assessment for patient-centered validation.Frontiers in aging · 2026Article
- Performance evaluation of 9 large language models in anticoagulation decision-making for nonvalvular atrial fibrillation.Research and practice in thrombosis and haemostasis · 2026Article
- A comparative study of the performance of different large language models in the Chinese National Pharmacist Licensing Examination.Frontiers in medicine · 2026Article
- Large Language Models for Preoperative Microvascular Invasion Prediction in Hepatocellular Carcinoma: A Multicenter Comparison with Radiologists and Treatment Outcomes.Journal of hepatocellular carcinoma · 2026Article
- Performance stability despite iteration: evaluating DeepSeek and ChatGPT on Chinese medical licensing examinations.Frontiers in medicine · 2026Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
3 authors.
Funding
Abstract
Large Language Models (LLMs) have a significant impact on medical education due to their advanced natural language processing capabilities. ChatGPT-4o (Chat Generative Pre-trained Transformer), a mainstream Western LLM, demonstrates powerful multimodal abilities. DeepSeek-R1, a newly released free and open-source LLM from China, demonstrates capabilities on par with ChatGPT-4o across various domains. This study aims to evaluate the performance of DeepSeek-R1 and ChatGPT-4o on the Chinese National Medical Licensing Examination (CNMLE) and explore the performance differences of LLMs from distinct linguistic environments in Chinese medical education. We evaluated both LLMs using 600 multiple-choice questions from the written part of 2024 CNMLE, covering four units. The questions were categorized into low- and high-difficulty groups according to difficulty. The primary outcome was the overall accuracy rate of each LLM. The secondary outcomes included accuracy within each of the four units and within the two difficulty-level groups. DeepSeek-R1 achieved a statistically significantly higher overall accuracy of 92.0% compared to ChatGPT-4o's 87.2% (P < 0.05). In the low-difficulty group, DeepSeek-R1 demonstrated an accuracy rate of 95.9%, which was significantly higher than ChatGPT-4o's 92.0% (P < 0.05). No statistically significant differences were observed between the models in any of the four units or in the high-difficulty group (P > 0.05). DeepSeek-R1 demonstrated a performance advantage on CNMLE.
Indexed as
Identifiers
40459679What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.