ArticleBMC medical education2025
The performance of ChatGPT on medical image-based assessments and implications for medical education.
Article in BMC medical education, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 12 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
12 citing papers in PubMed.
- Effect of large language model on diagnostic accuracy and clinical completeness among nephrology fellows managing transplant infection.International urology and nephrology · 2026Article
- Performance of multimodal large language models on image-based surgical anatomy, anatomical pathology, and radiology questions.Anatomical sciences education · 2026Article
- Large Language Models in German Continuing Medical Education Assessments: Protocol for a Fully Crossed Experimental Study.JMIR research protocols · 2026Article
- From Automated ECG Interpretation to Multimodal Cardiovascular Intelligence: The Evolution of Artificial Intelligence in Cardiovascular Medicine.Medical sciences (Basel, Switzerland) · 2026Review
- A 2-Tiered Rescue Protocol to Mitigate Difficulty-Based Failures of ChatGPT (GPT-5) and Gemini on the German M2 Medical Examination: Evaluation Study.JMIR formative research · 2026Article
- Evaluating ChatGPT-5 for Detection of Barrett's Esophagus and Grading of Esophagitis: A Multiclass Endoscopic Image Analysis.JGH open : an open access journal of gastroenterology and hepatology · 2026Article
- Artificial Intelligence in Medical Assessment: Reliability and Performance of Multimodal Large Language Models in a High-Stakes Licensing Examination.Behavioral sciences (Basel, Switzerland) · 2026Article
- Large Language Model-Assisted Point-in-Time Interpretation of Advanced Hemodynamics in Liver Transplant Recipients: A Pilot Evaluation of Content Quality and Safety.Journal of clinical medicine · 2026Article
- Comparative performance of ChatGPT-5 and DeepSeek on the Chinese ultrasound medicine senior professional title examination.Frontiers in digital health · 2026Article
- Generative artificial intelligence as an aid in interpreting thyroid FNA cytopathological images: Are we there yet?Cancer cytopathology · 2026Article
- Virtual patients and standardized patients combined training is associated with improved clinical reasoning among medical students.Frontiers in medicine · 2026Article
- Evaluation of ChatGPT-5 for Automated ASPECTS Assessment on Non-Contrast CT in Acute Ischemic Stroke.Diagnostics (Basel, Switzerland) · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundGenerative artificial intelligence (AI) tools like ChatGPT (OpenAI) have garnered significant attention for their potential in fields such as medical education; however, their performance of large language and vision models on medical test items involving images remains underexplored, limiting their broader educational utility. This study aims to evaluate the performance of GPT-4 and GPT-4 Omni (GPT-4o), accessed via the ChatGPT platform, on image-based United States Medical Licensing Examination (USMLE) sample items, to explore their implications for medical education.
methodsWe identified all image-based questions from the USMLE Step 1 and Step 2 Clinical Knowledge sample item sets. Prompt engineering techniques were applied to generate responses from GPT-4 and GPT-4o. Each model was independently tested, with accuracy calculated based on the proportion of correct answers. In addition, we explored the application of these models in case-based teaching scenarios involving medical images.
resultsA total of 38 image-based questions spanning multiple medical disciplines-including dermatology, cardiology, and gastroenterology-were included in the analysis. GPT-4 achieved an accuracy rate of 73.4% (95% CI, 57.0% to 85.5%), while GPT-4o outperformed it with an accuracy of 89.5% (95% CI, 74.4% to 96.1%), with a numerically higher accuracy but no statistically significant difference (P = 0.137). The two models showed substantial disagreement in their classification of question complexity. In exploratory case-based teaching scenarios, GPT-4o was able to analyze and revise incorrect responses with logical reasoning. Moreover, it demonstrated potential to assist educators in designing structured lesson plans focused on core clinical knowledge areas, though human oversight remained essential.
conclusionThis study demonstrates that GPT models can accurately answer image-based medical examination questions, with GPT-4o exhibiting numerically higher performance. Prompt engineering further enables their use in instructional planning. While these models hold promise for enhancing medical education, expert supervision remains critical to ensure the accuracy and reliability of AI-generated content.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.