ArticleJournal of medical systems2023
ChatGPT Performs on the Chinese National Medical Licensing Examination.
Article in Journal of medical systems, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 75 papers, 5 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
75 citing papers in PubMed, 5 syntheses or guidelines pooled it.
- Evaluating the Potential and Accuracy of ChatGPT-3.5 and 4.0 in Medical Licensing and In-Training Examinations: Systematic Review and Meta-Analysis.JMIR medical education · 2025Pooled it
- Performance of large language models in medical licensing examinations: a systematic review and meta-analysis.Journal of educational evaluation for health professions · 2025Pooled it
- The Accuracy and Capability of Artificial Intelligence Solutions in Health Care Examinations and Certificates: Systematic Review and Meta-Analysis.Journal of medical Internet research · 2024Pooled it
- Performance of ChatGPT-3.5 and GPT-4 in national licensing examinations for medicine, pharmacy, dentistry, and nursing: a systematic review and meta-analysis.BMC medical education · 2024Pooled it
- Performance of ChatGPT Across Different Versions in Medical Licensing Examinations Worldwide: Systematic Review and Meta-Analysis.Journal of medical Internet research · 2024Pooled it
- Performance of large language models in the Chinese National Medical Licensing Examinations and Beyond: Scoping review.PLOS digital health · 2026Article
- Large Language Model Performance on Multistep Clinical Cases: Comparative Study Across Question and Case Levels.JMIR medical education · 2026Article
- [Comparative study on the application of large language models in pre-exam learning for the Chinese dental licen-sing examination].Hua xi kou qiang yi xue za zhi = Huaxi kouqiang yixue zazhi = West China journal of stomatology · 2026Article
- Three large language models demonstrate competitive performance in Traditional Chinese Medicine national medical licensing examinations over two years.Scientific reports · 2026Article
- Multi-metric comparative evaluation of DeepSeek and ChatGPT in USMLE versus CNMLE for medical education.Scientific reports · 2026Article
- Performance Evaluation of Large Language Models in Multilingual Medical Multiple-Choice Questions: Mixed Methods Study.JMIR medical education · 2026Article
- Evaluation of the performance of four different large language models (ChatGPT, DeepSeek, Copilot, and Gemini) in answering oral, and maxillofacial radiology questions: pilot study.BMC oral health · 2026Article
- Comparing ChatGPT and Dental Students' Performance in an Introduction to Dental Anatomy Examination: A Cross-Sectional Study.European journal of dentistry · 2026Article
- Large Language Models Evaluation of Medical Licensing Examination Using GPT-4.0, ERNIE Bot 4.0, and GPT-4o.Bioengineering (Basel, Switzerland) · 2026Article
- Evaluation of DeepSeek-R1 and ChatGPT-4o on the Chinese national medical licensing examination: a multi-year comparative study.Scientific reports · 2026Article
- Performance stability despite iteration: evaluating DeepSeek and ChatGPT on Chinese medical licensing examinations.Frontiers in medicine · 2026Article
- Benchmarking large language models for medical education: performance on the clinical laboratory technician qualification examination.Frontiers in medicine · 2026Article
- Multiple Large Language Models' Performance on the Chinese Medical Licensing Examination: Quantitative Comparative Study.JMIR human factors · 2025Article
- Article
- Review
15 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
10 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
ChatGPT, a language model developed by OpenAI, uses a 175 billion parameter Transformer architecture for natural language processing tasks. This study aimed to compare the knowledge and interpretation ability of ChatGPT with those of medical students in China by administering the Chinese National Medical Licensing Examination (NMLE) to both ChatGPT and medical students. We evaluated the performance of ChatGPT in three years' worth of the NMLE, which consists of four units. At the same time, the exam results were compared to those of medical students who had studied for five years at medical colleges. ChatGPT's performance was lower than that of the medical students, and ChatGPT's correct answer rate was related to the year in which the exam questions were released. ChatGPT's knowledge and interpretation ability for the NMLE were not yet comparable to those of medical students in China. It is probable that these abilities will improve through deep learning.
Indexed as
Identifiers
37581690What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.