ArticleBMC medical education2025
Performance of artificial intelligence on Turkish dental specialization exam: can ChatGPT-4.0 and gemini advanced achieve comparable results to humans?
Article in BMC medical education, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 38 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
38 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Educational Applications of ChatGPT in University-Based Dental Education. A Systematic Review.European journal of dental education : official journal of the Association for Dental Education in Europe · 2026Pooled it
- Accuracy of large language models in the Turkish dental specialization examination (DUS): a multidimensional evaluation across disciplines and question formats.BMC oral health · 2026Article
- Evaluation of large language models in root resorption scenarios: an ESE-aligned comparative performance assessment.Odontology · 2026Article
- Accuracy and Temporal Consistency of ChatGPT and Gemini in Responding to Textbook and Patient-Oriented Dental Bleaching Questions: A Multi-Session Comparative Study.Journal of esthetic and restorative dentistry : official publication of the American Academy of Esthetic Dentistry ... [et al.] · 2026Article
- Article
- Bloom's taxonomy-based comparison of artificial intelligence and dental students in restorative dentistry.BMC medical education · 2026Article
- Benchmarking GPT-5, Gemini 2.5 Pro, Grok 4, and other LLMs on pediatric dentistry questions from a dental specialization exam.Scientific reports · 2026Article
- Large language models and generative artificial intelligence in endodontics: a scoping review.Odontology · 2026Review
- Performance of large language models on the Turkish Pharmacy Specialty Examination: a comparative analysis of accuracy, confidence, and readability.Scientific reports · 2026Article
- Cognitive-level analysis of dentomaxillofacial radiology questions in the Turkish dentistry specialization examination: a Bloom's revised taxonomy analysis.BMC oral health · 2026Article
- Artificial Intelligence in Medical Assessment: Reliability and Performance of Multimodal Large Language Models in a High-Stakes Licensing Examination.Behavioral sciences (Basel, Switzerland) · 2026Article
- Are chatbots reliable sources of information regarding fluoride in pediatric dentistry?BMC oral health · 2026Article
- Generative Artificial Intelligence-driven orthodontic education practices.BMC medical education · 2026Review
- Comparative diagnostic accuracy of multiple large language models in oral and maxillofacial radiology specialty examinations: a 13-year analysis of performance and topic trends.BMC oral health · 2026Article
- The effectiveness of large language models in dental specialty questions: a comparative study in the field of prosthodontics.BMC medical education · 2026Article
- Comparative performance evaluation of ChatGPT-4 Omni and Gemini Advanced in the Turkish Dentistry Specialization Exam.BMC medical education · 2026Article
- Can artificial intelligence chatbots think like dentists? A comparative analysis based on dental specialty examination questions in restorative dentistry.BMC oral health · 2026Article
- What the performance of multimodal LLMs on a national licensing exam teaches us about occupational therapy education.BMC medical education · 2026Article
- Article
- Accuracy and response repeatability of three large language models on undergraduate operative dentistry multiple-choice questions.Frontiers in dental medicine · 2026Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundAI-powered chatbots have spread to various fields including dental education and clinical assistance to treatment planning. The aim of this study is to assess and compare leading AI-powered chatbot performances in dental specialization exam (DUS) administered in Turkey and compare it with the best performer of that year.
methodsDUS questions for 2020 and 2021 were directed to ChatGPT-4.0 and Gemini Advanced individually. DUS questions were manually entered into AI-powered chatbot in their original form, in Turkish. The results obtained were compared with each other and the year's best performers. Candidates who score at least 45 points on this centralized exam are deemed to have passed and are eligible to select their preferred department and institution. The data was statistically analyzed using Pearson's chi-squared test (p < 0.05).
resultsChatGPT-4.0 received 83.3% correct response rate on the 2020 exam, while Gemini Advanced received 65% correct response rate. On the 2021 exam, ChatGPT-4.0 received 80.5% correct response rate, whereas Gemini Advanced received 60.2% correct response rate. ChatGPT-4.0 outperformed Gemini Advanced in both exams (p < 0.05). AI-powered chatbots performed worse in overall score (for 2020: ChatGPT-4.0, 65,5 and Gemini Advanced, 50.1; for 2021: ChatGPT-4.0, 65,6 and Gemini Advanced, 48.6) when compared to overall scores of the best performer of that year (68.5 points for year 2020 and 72.3 points for year 2021). This poor performance also includes the basic sciences and clinical sciences sections (p < 0.001). Additionally, periodontology was the clinical specialty in which both AI-powered chatbots achieved the best results, the lowest performance was determined in the endodontics and orthodontics.
conclusionAI-powered chatbots, namely ChatGPT-4.0 and Gemini Advanced, passed the DUS by exceeding the threshold score of 45. However, they still lagged behind the top performers of that year, particularly in basic sciences, clinical sciences, and overall score. Additionally, they exhibited lower performance in some clinical specialties such as endodontics and orthodontics.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.