ArticleClinical neuroradiology2024
Comparing the Diagnostic Performance of GPT-4-based ChatGPT, GPT-4V-based ChatGPT, and Radiologists in Challenging Neuroradiology Cases.
Article in Clinical neuroradiology, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 42 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
42 citing papers in PubMed, 1 synthesis or guideline pooled it.
- The impact of generative artificial intelligence on clinical skills and knowledge acquisition in medical undergraduates: a systematic review and meta-analysis.BMC medical education · 2026Pooled it
- Human-AI collaboration in large language model-assisted brain MRI differential diagnosis: a usability study.European radiology · 2025Trial
- Comparison of ChatGPT and Internet Research for Clinical Research and Decision-Making in Occupational Medicine: Randomized Controlled Trial.JMIR formative research · 2025Trial
- Diagnostic Accuracy of Multimodal Large Language Models (LLMs) for Detecting Tuberous Breast Deformity from Standardized Photographs: A Descriptive in Silico Study.Aesthetic plastic surgery · 2026Article
- Diagnostic accuracy and consistency of ChatGPT-4o in radiology: influence of image, clinical data, and answer options on performance.Diagnostic and interventional radiology (Ankara, Turkey) · 2026Article
- Large Language Models in the Radiology Workflow: Current Applications, Challenges, and Future Perspectives.Journal of the Korean Society of Radiology · 2026Review
- Evaluation of large language models in root resorption scenarios: an ESE-aligned comparative performance assessment.Odontology · 2026Article
- Do General-Purpose Multimodal Large Language Models Really See Radiologic Images or Rely on Text?Korean journal of radiology · 2026Article
- Evaluating the Accuracy and Diagnostic Reasoning of Multimodal Large Language Models in Interpreting Neuroradiology Cases FromKorean journal of radiology · 2026Article
- Impact of Large Language Model Assistance on Radiologists' Diagnostic Performance for Brain Tumors by Experience Level.Journal of clinical medicine · 2026Article
- Independent and collaborative performance of large language models and healthcare professionals in diagnosis and triage.NPJ digital medicine · 2026Article
- Large Language Models for Psychiatric Diagnosis Based on Multicenter Real-World Clinical Records: Comparative Study.JMIR medical informatics · 2026Article
- Assessing GPT-4o in cataract surgery decision-making: appropriateness, consistency, and clinical implications.Frontiers in artificial intelligence · 2026Article
- Diagnostic Accuracy of GPT-4 With Vision in Neuroradiology Board-Style Exam Questions: Cross-Sectional Case-Based Study.JMIR neurotechnology · 2026Article
- Performance of artificial intelligence chatbots in the diagnosis and management of simulated dental trauma cases: an evaluation based on IADT guidelines.Clinical oral investigations · 2025Article
- Trends and Trajectories in the Rise of Large Language Models in Radiology: Scoping Review.JMIR medical informatics · 2025Article
- Comparative analysis of generic vision-language models in detecting and diagnosing inherited retinal diseases using fundus photographs.Eye (London, England) · 2025Article
- Evaluating Large Language Models in Interpreting MRI Reports and Recommending Treatment for Vestibular Schwannoma.Diagnostics (Basel, Switzerland) · 2025Article
- Article
- Diagnostic Performance of GPT-4o Compared to Radiology Residents in Emergency Abdominal Tomography Cases.Tomography (Ann Arbor, Mich.) · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
11 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
purposeTo compare the diagnostic performance among Generative Pre-trained Transformer (GPT)-4-based ChatGPT, GPT‑4 with vision (GPT-4V) based ChatGPT, and radiologists in challenging neuroradiology cases.
methodsWe collected 32 consecutive "Freiburg Neuropathology Case Conference" cases from the journal Clinical Neuroradiology between March 2016 and December 2023. We input the medical history and imaging findings into GPT-4-based ChatGPT and the medical history and images into GPT-4V-based ChatGPT, then both generated a diagnosis for each case. Six radiologists (three radiology residents and three board-certified radiologists) independently reviewed all cases and provided diagnoses. ChatGPT and radiologists' diagnostic accuracy rates were evaluated based on the published ground truth. Chi-square tests were performed to compare the diagnostic accuracy of GPT-4-based ChatGPT, GPT-4V-based ChatGPT, and radiologists.
resultsGPT‑4 and GPT-4V-based ChatGPTs achieved accuracy rates of 22% (7/32) and 16% (5/32), respectively. Radiologists achieved the following accuracy rates: three radiology residents 28% (9/32), 31% (10/32), and 28% (9/32); and three board-certified radiologists 38% (12/32), 47% (15/32), and 44% (14/32). GPT-4-based ChatGPT's diagnostic accuracy was lower than each radiologist, although not significantly (all p > 0.07). GPT-4V-based ChatGPT's diagnostic accuracy was also lower than each radiologist and significantly lower than two board-certified radiologists (p = 0.02 and 0.03) (not significant for radiology residents and one board-certified radiologist [all p > 0.09]).
conclusionWhile GPT-4-based ChatGPT demonstrated relatively higher diagnostic performance than GPT-4V-based ChatGPT, the diagnostic performance of GPT‑4 and GPT-4V-based ChatGPTs did not reach the performance level of either radiology residents or board-certified radiologists in challenging neuroradiology cases.
Indexed as
Identifiers
38806794What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.