ArticleBMC medical education2026
Performance of ChatGPT in Israeli Arabic-language OBGYN national medical licensure exam.
Article in BMC medical education, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
0 citing papers in PubMed.
No citing paper in PubMed yet.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundPrevious studies of ChatGPT performance in the field of medical exams have reached contradictory results. The performance of ChatGPT in languages other than English, including Arabic, which is the official language of medical education and practice in many countries, has yet to be explored. We aim to evaluate the performance of ChatGPT in Arabic-language Israeli OBGYN medical licensure exams for foreign university alumni.
methodsWe conducted a performance study using a consecutive sample of text-based multiple-choice questions, originated from authentic Arabic-language Israeli OBGYN medical licensure exams for foreign university alumni. ChatGPT-3.5 (using a newly created account) answered all questions in Arabic. We compared the performance of ChatGPT including in the different fields of the exam; Obstetrics, Reproductive medicine and Infertility, Gynecology and Gynecologic Oncology, and also compared ChatGPT Arabic performance vs. previously published English medical tests.
resultsOverall, 123 authentic questions were analyzed. ChatGPT correctly answered 54 questions (43.9%, 95% CI: 35.1% – 52.7%) and reached a score below 50%. There was no difference in ChatGPT performance in the four different subjects of the exam: Gynecologic Oncology (61.5%, 95% CI: 35.1% – 87.9%), Gynecology (44.0%, 95% CI: 24.5% – 63.5%), Obstetrics (42.3%, 95% CI: 28.9% – 55.7%), Reproductive medicine and Infertility (39.4%, 95% CI: 22.7% – 56.1%), p = .579. In a comparison to ChatGPT performance in 9,091 English language questions in the field of medicine, the performance of Arabic ChatGPT was lower (43.9% in Arabic vs. 60.7% in English, p < .001).
conclusionsChatGPT-3.5 answered correctly approximately 44% of Arabic OBGYN medical licensure exam questions. At the time of writing of this manuscript, considering the results of our analysis, ChatGPT-3.5 cannot be considered a reliable primary tool for exam preparation in Arabic. Further research and efforts should be made to improve ChatGPT performance in other languages besides English especially Arabic.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.