SynthesisAnatomical sciences education2026
The performance of ChatGPT and other large language models on multiple-choice questions in biomedical disciplines: A meta-analysis.
Synthesis in Anatomical sciences education, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 1 paper.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
1 citing paper in PubMed.
- Comparative evaluation of radiological anatomy knowledge and accuracy of ChatGPT-5, Gemini 2.5, and Grok 4 across normal and thinking modes.Anatomical sciences education · 2026Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
27 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
While large language models (LLMs) have shown promise as learning tools for medical education, their reported accuracy on multiple-choice questions (MCQs) varies widely across studies, necessitating synthesis. This meta-analysis synthesizes LLM accuracy on text-based MCQs from biomedical disciplines and USMLE Step 1-level content and explores study characteristics that moderate LLM performance. Studies published between January 1, 2022, and August 5, 2025, were identified using seven databases. Titles and abstracts were screened against eligibility criteria, which included testing LLM performance on text-based MCQs in English. Extracted data included accuracy rates (correct responses/total questions) and study characteristics. A random-effects proportional meta-analysis was used to calculate the pooled accuracy of LLM versions across studies, and a meta-regression tested the effects of study characteristics on accuracy. The final analysis included 41 articles comprising 207 proportions. Newer OpenAI models (GPT-4o through GPT o1 = 90%; GPT-4 = 82%) demonstrated significantly higher accuracy than earlier versions (GPT-3/3.5 = 59%; p < 0.001). Model version explained 56% of the total variance (p < 0.001). The pooled accuracy of newer Anthropic (Claude 3 through 3.7 = 86%) and DeepSeek (R1 = 86%) versions fell within the range of performance for the newer OpenAI versions, while newer Google (Gemini series = 76%) and Microsoft (Copilot = 69%) versions demonstrated lower performance. A discipline-level analysis demonstrated that OpenAI's GPT-4 through GPT-o1 models achieved pooled accuracy rates between 83 and 87% for most disciplines, including the anatomical sciences. These findings demonstrate that newer LLMs are achieving higher accuracy than older models on biomedical and Step 1-level MCQs, supporting their potential integration as supplementary study tools in foundational biomedical education.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.