ArticleDigital health
Exploring the proficiency of ChatGPT-4: An evaluation of its performance in the Taiwan advanced medical licensing examination.
Article in Digital health. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers, 3 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
16 citing papers in PubMed, 3 syntheses or guidelines pooled it.
- Evaluating the Potential and Accuracy of ChatGPT-3.5 and 4.0 in Medical Licensing and In-Training Examinations: Systematic Review and Meta-Analysis.JMIR medical education · 2025Pooled it
- Performance of large language models in medical licensing examinations: a systematic review and meta-analysis.Journal of educational evaluation for health professions · 2025Pooled it
- Performance of ChatGPT Across Different Versions in Medical Licensing Examinations Worldwide: Systematic Review and Meta-Analysis.Journal of medical Internet research · 2024Pooled it
- Artificial intelligence performance in the emergency medicine subspecialty examination conducted in Türkiye.Medicine · 2026Article
- Benchmarking Large Language Models on the Taiwan Neurology Board Examinations (2018-2024): A Comparative Evaluation of GPT-4o, GPT-o1, DeepSeek-V3, and DeepSeek-R1.Bioengineering (Basel, Switzerland) · 2026Article
- Can ChatGPT-5 educate the public about vasectomy?: a Google Trends-based expert panel assessment.Frontiers in digital health · 2026Article
- Article
- Review
- Evaluating performance of large language models for atrial fibrillation management using different prompting strategies and languages.Scientific reports · 2025Article
- Evaluation and comparison of large language models' responses to questions related optic neuritis.Frontiers in medicine · 2025Article
- Advancing medical AI: GPT-4 and GPT-4o surpass GPT-3.5 in Taiwanese medical licensing exams.PloS one · 2025Article
- Supporting postgraduate exam preparation with large language models: implications for traditional Chinese medicine education.Frontiers in medicine · 2025Article
- How well do large language model-based chatbots perform in oral and maxillofacial radiology?Dento maxillo facial radiology · 2024Article
- Federated Learning in Glaucoma: A Comprehensive Review and Future Perspectives.Ophthalmology. GlaucomaReview
- ChatGPT performance in answering medical residency questions in nephrology: a pilot study in Brazil.Jornal brasileiro de nefrologiaArticle
- Assessing AI efficacy in medical knowledge tests: A study using Taiwan's internal medicine exam questions from 2020 to 2023.Digital healthArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Background: Taiwan is well-known for its quality healthcare system. The country's medical licensing exams offer a way to evaluate ChatGPT's medical proficiency. Methods: We analyzed exam data from February 2022, July 2022, February 2023, and July 2033. Each exam included four papers with 80 single-choice questions, grouped as descriptive or picture-based. We used ChatGPT-4 for evaluation. Incorrect answers prompted a "chain of thought" approach. Accuracy rates were calculated as percentages. Results: ChatGPT-4's accuracy in medical exams ranged from 63.75% to 93.75% (February 2022-July 2023). The highest accuracy (93.75%) was in February 2022's Medicine Exam (3). Subjects with the highest misanswered rates were ophthalmology (28.95%), breast surgery (27.27%), plastic surgery (26.67%), orthopedics (25.00%), and general surgery (24.59%). While using "chain of thought," the "Accuracy of (CoT) prompting" ranged from 0.00% to 88.89%, and the final overall accuracy rate ranged from 90% to 98%. Conclusion: ChatGPT-4 succeeded in Taiwan's medical licensing exams. With the "chain of thought" prompt, it improved accuracy to over 90%.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.