ArticleJAMIA open2025
Comparative analysis of large language models in clinical diagnosis: performance evaluation across common and complex medical cases.
Article in JAMIA open, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 18 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
18 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Accuracy of Large Language Models in Answering Dental Examination Questions: A Systematic Review and Meta-Analysis.International dental journal · 2026Pooled it
- Large Language Models in Perioperative Antithrombotic Management: A Blinded Scenario-Based Expert Evaluation.Diagnostics (Basel, Switzerland) · 2026Article
- Alignment Between Large Language Models and Consensus in Endoscopic Spinal Disc Surgery: A Comparative Analysis Against the Neurocore-SENSED.JOR spine · 2026Article
- Diagnostic Performance and Workup Efficiency of Large Language Models in Secondary Hypertension: A Blinded Comparative Study.Diagnostics (Basel, Switzerland) · 2026Article
- Symptom-Only Localization of Brainstem Ischemia Using Large Language Models Versus Neurologists in Diffusion-Weighted Imaging-Positive Cases: Retrospective Single-Center Study.JMIR formative research · 2026Article
- Multi-model large-scale AI framework for avian influenza surveillance and preparedness: Harnessing large language models to enhance risk communication, real-time decision support, and public health response strategies.One health (Amsterdam, Netherlands) · 2026Article
- Gender-Attributed Persona Prompts and the Diagnostic Accuracy of Proprietary and Open-Weight Large Language Models in Chagas Disease and Visceral Leishmaniasis: A Paired Experimental Study.Healthcare (Basel, Switzerland) · 2026Article
- Agentic GPT-5.0 system outperforms standard large language models and human experts in critical care clinical decision-making: a simulation study.BMC medical informatics and decision making · 2026Article
- Evaluating large language models for pharmacotherapy simulations: a mixed-methods study.NPJ digital medicine · 2026Article
- Accuracy and Knowledge Base Evaluation of ChatGPT-4o, Gemini-2.0-Flash, and DeepSeek-V3 in Metabolic and Bariatric Surgery: an Expert-Rated Blinded Study.Obesity surgery · 2026Article
- Generative AI vs web search for patient education: a comparative evaluation of OSA information quality.Journal of clinical sleep medicine : JCSM : official publication of the American Academy of Sleep Medicine · 2026Article
- Evaluating large language models for endodontic case difficulty assessment: accuracy, consistency, temporal stability, and agreement.BMC oral health · 2026Article
- Generative Artificial Intelligence in Neonatal Diagnostic Reasoning: Promise, Prudence, and the Path Ahead.Indian journal of pediatrics · 2026Article
- AI-Powered Clinical Decision Support in Dentistry: Comparative Evaluation of Large Language Models for Oral Medicine and Periodontal Diagnosis.Medical science monitor : international medical journal of experimental and clinical research · 2026Article
- Evaluating Large Language Models for Diagnostic Accuracy and Health Information Quality in Oral Mucosal Diseases.Journal of personalized medicine · 2026Article
- Artificial intelligence based resuscitation simulation: a pilot study of a novel approach to team leadership training.BMC medical education · 2026Article
- A novel evaluation benchmark for medical LLMs illuminating safety and effectiveness in clinical domains.NPJ digital medicine · 2025Article
- Leveraging ChatGPT and explainable AI for enhancing clinical decision support.Scientific reports · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Objectives: This study aimed to systematically evaluate and compare the diagnostic performance of leading large language models (LLMs) in common and complex clinical scenarios, assessing their potential for enhancing clinical reasoning and diagnostic accuracy in authentic clinical decision-making processes. Materials and Methods: Diagnostic capabilities of advanced LLMs (Anthropic's Claude, OpenAI's GPT variants, Google's Gemini) were assessed using 60 common cases and 104 complex, real-world cases from Clinical Problem Solvers' morning rounds. Clinical details were disclosed in stages, mirroring authentic clinical decision-making. Models were evaluated on primary and differential diagnosis accuracy at each stage. Results: Advanced LLMs showed high diagnostic accuracy (>90%) in common scenarios, with Claude 3.7 achieving perfect accuracy (100%) in certain conditions. In complex cases, Claude 3.7 achieved the highest accuracy (83.3%) at the final diagnostic stage, significantly outperforming smaller models. Smaller models notably performed well in common scenarios, matching the performance of larger models. Discussion: This study evaluated leading LLMs for diagnostic accuracy using staged information disclosure, mirroring real-world practice. Notably, Claude 3.7 Sonnet was the top performer. Employing a novel LLM-based evaluation method for large-scale analysis, the research highlights artificial intelligence's (AI's) potential to enhance diagnostics. It underscores the need for useful frameworks to translate accuracy into clinical impact and integrate AI into medical education. Conclusion: Leading LLMs show remarkable diagnostic accuracy in diverse clinical cases. To fully realize their potential for improving patient care, we must now focus on creating practical implementation frameworks and translational research to integrate these powerful AI tools into medicine.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.