ArticleScientific reports2025
Evaluating the performance of GPT-3.5, GPT-4, and GPT-4o in the Chinese National Medical Licensing Examination.
Article in Scientific reports, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 25 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
25 citing papers in PubMed.
- Performance of large language models in the Chinese National Medical Licensing Examinations and Beyond: Scoping review.PLOS digital health · 2026Article
- Large Language Model Performance on Multistep Clinical Cases: Comparative Study Across Question and Case Levels.JMIR medical education · 2026Article
- Performance of Seven Large Language Models on Anatomy Examination Questions.Clinical anatomy (New York, N.Y.) · 2026Article
- Performance of multimodal large language models on image-based surgical anatomy, anatomical pathology, and radiology questions.Anatomical sciences education · 2026Article
- Diagnostic Performance of Contemporary Large Language Models on Free-Text Histopathologic Descriptions in Oral and Maxillofacial Pathology.Head and neck pathology · 2026Article
- Guideline-based clinical reasoning in periodontology education: a comparative study of residents and large language models.BMC medical education · 2026Article
- Evaluation of ChatGPT-4o in oral and maxillofacial surgery examinations: a comparative study of performance on U.S. dental decks and chinese dental licensing examination practice questions.BMC oral health · 2026Article
- A Dataset for Evaluating Large Language Models on Chinese National Medical Licensing Examinations.Scientific data · 2026Article
- Artificial Intelligence Literacy in Intensive Care: From Algorithmic Fluency to Clinical Accountability.Indian journal of critical care medicine : peer-reviewed, official publication of Indian Society of Critical Care Medicine · 2026Article
- Article
- Large Language Models Evaluation of Medical Licensing Examination Using GPT-4.0, ERNIE Bot 4.0, and GPT-4o.Bioengineering (Basel, Switzerland) · 2026Article
- Evaluation of DeepSeek-R1 and ChatGPT-4o on the Chinese national medical licensing examination: a multi-year comparative study.Scientific reports · 2026Article
- Performance stability despite iteration: evaluating DeepSeek and ChatGPT on Chinese medical licensing examinations.Frontiers in medicine · 2026Article
- Diagnostic accuracy and repeatability of ChatGPT using textual and radiographic data in reversible pulpitis: a retrospective diagnostic study.Frontiers in bioinformatics · 2026Article
- Comprehensive Evaluation of Large Language Models on Four Core Medical School Courses: A Cross-Sectional Comparative Study.Advances in medical education and practice · 2026Article
- From accuracy to robustness: a comparative study of five advanced large language models on the Chinese dental licensing examination.BMC oral health · 2025Article
- Teaching Clinical Reasoning in Health Care Professions Learners Using AI-Generated Script Concordance Tests: Mixed Methods Formative Evaluation.JMIR formative research · 2025Article
- LLMs outperform outsourced human coders on complex textual analysis.Scientific reports · 2025Article
- Comparative performance of Chinese and international large language models on the Chinese radiology attending physician qualification examination.Scientific reports · 2025Article
- [Application of large language models in health education for patients with pediatric cataract].Zhong nan da xue xue bao. Yi xue ban = Journal of Central South University. Medical sciences · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
10 authors.
Funding
Abstract
This study aims to compare and evaluate the performance of GPT-3.5, GPT-4, and GPT-4o in the 2020 and 2021 Chinese National Medical Licensing Examination (NMLE), exploring their potential value in medical education and clinical applications. Six hundred original test questions from the 2020 and 2021 NMLE (covering five types of questions) were selected and input into GPT-3.5, GPT-4, and GPT-4o for response. The accuracy of the models across different question types and units was recorded and analyzed. Statistical methods were employed to compare the performance differences among the three models. GPT-4o demonstrated significantly higher overall accuracy than GPT-4 and GPT-3.5 (P < 0.001). In the 2020 and 2021 exams, GPT-4o achieved accuracy rates of 84.2% and 88.2%, respectively, with the highest accuracy observed in questions related to the digestive system (Unit 3), reaching 94.75%. GPT-4 showed moderate performance, while GPT - 3.5 had the lowest accuracy. Additionally, GPT-4o exhibited a clear advantage in complex question formats, such as case analysis questions (A3/A4 type) and standard matching questions (B1 type). GPT-4o outperformed its predecessors in the NMLE, demonstrating exceptional comprehension and problem-solving abilities in non-English medical examinations. This study provides important insights into the application and promotion of generative AI in medical education and clinical practice.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.