ArticleJournal of medical Internet research2024
ChatGPT With GPT-4 Outperforms Emergency Department Physicians in Diagnostic Accuracy: Retrospective Analysis.
Article in Journal of medical Internet research, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT07632859 (Diagnostic Accuracy of Two Large Language Models Against a Blinded Specialist Consensus Standard in Turkish Emergency Department Notes), which is not on this map. Cited by 54 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Diagnostic Accuracy of Two Large Language Models Against a Blinded Specialist Consensus Standard in Turkish Emergency Department Notes: A Retrospective Study of 600 Cases
Who cites it
54 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Artificial intelligence in orthopaedic education, training and research: a systematic review.BMC medical education · 2025Pooled it
- A systematic review of large language model (LLM) evaluations in clinical medicine.BMC medical informatics and decision making · 2025Pooled it
- Natural Language Processing for Risk Stratification of Emergency Department Patients with Chest Pain.CJC open · 2026Article
- Comparative Expert Evaluation of Multimodal Large Language Models for Pediatric Rash Diagnosis: Clinical Utility, Safety, Information Quality, and Readability.Children (Basel, Switzerland) · 2026Article
- Should I and Can I Use AI for Forensic Psychiatry Report Writing?JMIR mental health · 2026Article
- ECG-based detection of occlusion myocardial infarction: a dedicated deep neural network versus multimodal large language models and physicians - a retrospective diagnostic accuracy study.BMC emergency medicine · 2026Article
- OntoCodex: a multi-agent biomedical ontology enrichment framework.npj health systems · 2026Article
- Psychological Risk Assessment in Plastic Surgery via a DeepSeek Large Language Model: A Retrospective Cohort Study.Aesthetic plastic surgery · 2026Article
- Incidence, contributing factors, and predictors of diagnostic errors in medical inpatients: A retrospective cohort study.Journal of hospital medicine · 2026Article
- Clinical Applications, Opportunities, and Implementation Challenges of AI in Emergency Medicine: A Narrative Review.Cureus · 2026Review
- Diagnostic capability of large language models in critically ill patients: a prospective single-centre study comparing ChatGPT, Claude, and Gemini with emergency physicians.BMC emergency medicine · 2026Observational
- Assessing AI and Neurologist Diagnostic Reasoning Against Neuropathological Ground Truth.medRxiv : the preprint server for health sciences · 2026Article
- Performance of DeepSeek V3.2 and ChatGPT 5.1 in Musculoskeletal Triage and Differential Diagnosis of Outpatients With Low Back Pain: Multidimensional Comparative Study.Journal of medical Internet research · 2026Article
- Care-pathway decision support improves non-specialist vet clinical decision making in dogs with myelopathy.Journal of veterinary internal medicine · 2026Article
- Comparative evaluation of seven large language models in providing home phototherapy care guidance for neonatal hyperbilirubinemia.Translational pediatrics · 2026Article
- Retrieval-augmented generative AI enhances clinical reasoning in odontogenic sinusitis versus maxillary sinus mucositis.European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery · 2026Article
- Comparison of Intern Doctors and ChatGPT in Emergency Cases Assessment.Journal of clinical practice and research · 2026Article
- Performance of Large Language Models on Exam-style Questions and Case Challenges Across Varying Levels of Complexity.Journal of medical systems · 2026Article
- ChatGPT in precision medicine.APL bioengineering · 2026Review
- Evaluation of Online Large Language Models as Decision Support for the Management of Abdominal Pain in the Emergency Department: A Pilot Study.Deutsches Arzteblatt international · 2026Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundOpenAI's ChatGPT is a pioneering artificial intelligence (AI) in the field of natural language processing, and it holds significant potential in medicine for providing treatment advice. Additionally, recent studies have demonstrated promising results using ChatGPT for emergency medicine triage. However, its diagnostic accuracy in the emergency department (ED) has not yet been evaluated.
objectiveThis study compares the diagnostic accuracy of ChatGPT with GPT-3.5 and GPT-4 and primary treating resident physicians in an ED setting.
methodsAmong 100 adults admitted to our ED in January 2023 with internal medicine issues, the diagnostic accuracy was assessed by comparing the diagnoses made by ED resident physicians and those made by ChatGPT with GPT-3.5 or GPT-4 against the final hospital discharge diagnosis, using a point system for grading accuracy.
resultsThe study enrolled 100 patients with a median age of 72 (IQR 58.5-82.0) years who were admitted to our internal medicine ED primarily for cardiovascular, endocrine, gastrointestinal, or infectious diseases. GPT-4 outperformed both GPT-3.5 (P<.001) and ED resident physicians (P=.01) in diagnostic accuracy for internal medicine emergencies. Furthermore, across various disease subgroups, GPT-4 consistently outperformed GPT-3.5 and resident physicians. It demonstrated significant superiority in cardiovascular (GPT-4 vs ED physicians: P=.03) and endocrine or gastrointestinal diseases (GPT-4 vs GPT-3.5: P=.01). However, in other categories, the differences were not statistically significant.
conclusionsIn this study, which compared the diagnostic accuracy of GPT-3.5, GPT-4, and ED resident physicians against a discharge diagnosis gold standard, GPT-4 outperformed both the resident physicians and its predecessor, GPT-3.5. Despite the retrospective design of the study and its limited sample size, the results underscore the potential of AI as a supportive diagnostic tool in ED settings.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.