ArticleInternational dental journal2025
Accuracy of Large Language Models for Infective Endocarditis Prophylaxis in Dental Procedures.
Article in International dental journal, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 19 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
19 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Explainable Artificial Intelligence in Dentistry: A Systematic Review of Its Trust and Translation.International dental journal · 2026Pooled it
- Comparative Benchmark of Seven Large Language Models for Traumatic Dental Injury Knowledge.European journal of dentistry · 2026Article
- Do large language models differ in their pharmacology-related response quality for oral and maxillofacial surgery? a blinded expert benchmark study.BMC oral health · 2026Article
- How Reliably Do Large Language Models Reproduce Vital Pulp Therapy Guidelines? A Mixed-Effects Evaluation of Guideline-Concordance and Error Directionality.Healthcare (Basel, Switzerland) · 2026Article
- Comparative evaluation of artificial intelligence language models on knowledge of dental antibiotic use.BMC oral health · 2026Article
- Large Language Models in Infectious Diseases: A Systemic Review.Research square · 2026Article
- Evaluation of the performance of four different large language models (ChatGPT, DeepSeek, Copilot, and Gemini) in answering oral, and maxillofacial radiology questions: pilot study.BMC oral health · 2026Article
- A Survey on Medical Competence Evaluation Benchmarks for Large Language Models.Health care science · 2026Review
- Evaluating Retrieval-Augmented Generation-Large Language Models for Infective Endocarditis Prophylaxis: Clinical Accuracy and Efficiency.International dental journal · 2026Article
- A comparative analysis of the performance of large Language models in the dentistry specialty examination.Scientific reports · 2026Article
- Multi-Regional deep learning models for identifying dental restorations and prosthesis in panoramic radiographs.BMC oral health · 2025Article
- Mapping Review of the Correlations Between Periodontitis, Dental Caries, and Endocarditis.Dentistry journal · 2025Review
- Assessment of ChatGPT's adherence to evidence-based clinical practice guidelines for plantar fasciitis management.Journal of orthopaedic surgery and research · 2025Article
- Large Language Models in Action: Supporting Clinical Evaluation in an Infectious Disease Unit.Healthcare (Basel, Switzerland) · 2025Article
- The Transformative Role of Artificial Intelligence in Dentistry: A Comprehensive Overview. Part 1: Fundamentals of AI, and its Contemporary Applications in Dentistry.International dental journal · 2025Review
- Chat Generative Pre-Trained Transformer (ChatGPT) in Oral and Maxillofacial Surgery: A Narrative Review on Its Research Applications and Limitations.Journal of clinical medicine · 2025Review
- Automatic extraction of SmPC document for IDMP data model construction using foundation LLM and RAG: a preliminary experiment for pharmaceutical regulatory affairs.Frontiers in medicine · 2025Article
- Article
- Accuracy of large language models in interpreting urological clinical guidelines: a comparative study with expert evaluation.Therapeutic advances in urologyArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
purposeInfective endocarditis (IE) is a serious, life-threatening condition requiring antibiotic prophylaxis for high-risk individuals undergoing invasive dental procedures. As LLMs are rapidly adopted by dental professionals for their efficiency and accessibility, assessing their accuracy in answering critical questions about antibiotic prophylaxis for IE prevention is crucial.
methodsTwenty-eight true/false questions based on the 2021 American Heart Association (AHA) guidelines for IE were posed to 7 popular LLMs. Each model underwent five independent runs per question using two prompt strategies: a pre-prompt as an experienced dentist and without a pre-prompt. Inter-model comparisons utilised the Kruskal-Wallis test, followed by post-hoc pairwise comparisons using Prism 10 software.
resultsSignificant differences in accuracy were observed among the LLMs. All LLMs had a narrower confidence interval with a pre-prompt, and most, except Claude 3 Opus, showed improved performance. GPT-4o had the highest accuracy (80% with a pre-prompt, 78.57% without), followed by Gemini 1.5 Pro (78.57% and 77.86%) and Claude 3 Opus (75.71% and 77.14%). Gemini 1.5 Flash had the lowest accuracy (68.57% and 63.57%). Without a pre-prompt, Gemini 1.5 Flash's accuracy was significantly lower than Claude 3 Opus, Gemini 1.5 Pro, and GPT-4o. With a pre-prompt, Gemini 1.5 Flash and Claude 3.5 were significantly less accurate than Gemini 1.5 Pro and GPT-4o. None of the LLMs met the commonly used benchmark scores. All models provided both correct and incorrect answers randomly, except Claude 3.5 Sonnet with a pre-prompt, which consistently gave incorrect answers to eight questions across five runs.
conclusionLLMs like GPT-4o show promise for retrieving AHA-IE guideline information, achieving up to 80% accuracy. However, complex medical questions may still pose a challenge. Pre-prompts offer a potential solution, and domain-specific training is essential for optimizing LLM performance in healthcare, especially with the emergence of models with increased token limits.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.