ArticleBMC medical education2025
Evaluating large language models using national endodontic specialty examination questions: are they ready for real-world dentistry?
Article in BMC medical education, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 10 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
10 citing papers in PubMed.
- Comparative Evaluation of Large Language Models in Responding to Common Misconceptions Regarding Periodontal Disease and Oral Hygiene.Diagnostics (Basel, Switzerland) · 2026Article
- Accuracy of large language models in the Turkish dental specialization examination (DUS): a multidimensional evaluation across disciplines and question formats.BMC oral health · 2026Article
- Bloom's taxonomy-based comparison of artificial intelligence and dental students in restorative dentistry.BMC medical education · 2026Article
- How Reliably Do Large Language Models Reproduce Vital Pulp Therapy Guidelines? A Mixed-Effects Evaluation of Guideline-Concordance and Error Directionality.Healthcare (Basel, Switzerland) · 2026Article
- Comparative analysis of the performance of artificial intelligence language models on Turkish dental specialty examination questions.BMC oral health · 2026Article
- Comparative diagnostic accuracy of multiple large language models in oral and maxillofacial radiology specialty examinations: a 13-year analysis of performance and topic trends.BMC oral health · 2026Article
- A comparative analysis of the performance of large Language models in the dentistry specialty examination.Scientific reports · 2026Article
- Comparative performance evaluation of ChatGPT-4 Omni and Gemini Advanced in the Turkish Dentistry Specialization Exam.BMC medical education · 2026Article
- Artificial intelligence-powered chatbots' responses to orthodontic questions from the dentistry specialization examination: Accuracy and source evaluation.Journal of dental sciences · 2026Article
- Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundLarge Language Models (LLMs) are artificial intelligence (AI) systems that simulate human language processing through deep learning techniques and neural networks. They are increasingly utilized for clinical decision support, student training, and enhancing educational processes. However, the reliability of AI models, especially in answering various types of questions, remains a point of debate. Standard multiple-choice questions (MCQs) involve selecting one correct answer from five options, whereas combination-type MCQs (C-MCQs) identify all correct statements among several alternatives. This study aims to evaluate and compare the performance of various LLMs in answering MCQs and C-MCQs in endodontics.
methodsA total of 151 endodontic questions were identified through a comprehensive review of publicly available Dentistry Specialty Exams in Turkey conducted since 2012. The questions were presented to eight LLMs (ChatGPT-4o, ChatGPT-4, Gemini 1.5 Flash, Gemini 1.5 Pro, Gemini 2.0 Flash, Copilot, Deepseek-V3, and Qwen2.5-Max) in Turkish. Accuracy rates for both MCQs and C-MCQs were statistically analyzed using SPSS v23 (p < 0.05).
resultsChatGPT-4o achieved the highest overall accuracy rate (81.5%), while Gemini 1.5 Flash had the lowest (57%). In standard MCQs, ChatGPT-4o significantly outperformed the other models (p < 0.001), but in C-MCQs, no significant difference was observed between the models (p = 0.179). Across all models, accuracy rates for C-MCQs were significantly lower than for MCQs (p < 0.05). Deepseek-V3 maintained a more balanced performance across question types than the other models.
conclusionsLLMs show promising potential as educational tools in endodontics. However, their accuracy varies by question type and model. They can support student learning and clinical decision-making but cannot yet be considered a fully reliable standalone source in endodontics.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.