ArticleJMIR medical informatics2024
Evaluating ChatGPT-4's Diagnostic Accuracy: Impact of Visual Data Integration.
Article in JMIR medical informatics, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 27 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
27 citing papers in PubMed.
- Evaluating the Performance of ChatGPT-4V in Detecting Inflammatory Magnetic Resonance Imaging Findings of Sacroiliitis: Potentials, Challenges, and Limitations.Journal of imaging informatics in medicine · 2026Article
- Article
- Do Multimodal Vision-Language Models Enhance the Medical Diagnostic Process? A Systematic Review.Healthcare (Basel, Switzerland) · 2026Review
- Clinical Implications of ChatGPT-assisted Multimodal Pre-operative Assessment in Elderly Pertrochanteric Fracture Patients: An Exploratory Study.Journal of orthopaedic case reports · 2026Article
- ChatGPT, Claude Sonnet, and Grok Display Similarly Low Rates of Accuracy in Identifying Image-Based Orthopaedic Sports Pathologies.Arthroscopy, sports medicine, and rehabilitation · 2026Article
- Report Generation System for Slit-Lamp Image Interpretation Using Vision-Language Models.Ophthalmology and therapy · 2026Article
- Pricing models for diagnostic AI based on qualitative insights from healthcare decision makers.NPJ digital medicine · 2026Article
- Diagnostic Accuracy of GPT-4 With Vision in Neuroradiology Board-Style Exam Questions: Cross-Sectional Case-Based Study.JMIR neurotechnology · 2026Article
- Observational
- Performance of ChatGPT-5 in Diagnosing Fractures on Proximal Humerus and Intertrochanteric Femur X-Rays.Journal of orthopaedic case reports · 2026Article
- AI-assisted sentinel lymph node examination and metastatic detection in breast cancer: the potential of ChatGPT for digital pathology research.Pathologica · 2025Article
- Seeing Beyond the Microscope: Artificial Intelligence and Fluorescence Confocal Digital Imaging in Pediatric Surgical Pathology.Children (Basel, Switzerland) · 2025Article
- Comparative Assessment of Large Language Models in Optics and Refractive Surgery: Performance on Multiple-Choice Questions.Vision (Basel, Switzerland) · 2025Article
- Diagnostic capabilities of large language models in the detection of scaphoid fractures in the emergency department.Ulusal travma ve acil cerrahi dergisi = Turkish journal of trauma & emergency surgery : TJTES · 2025Observational
- ChatGPT-4's Level of Dermatological Knowledge Based on Board Examination Review Questions and Bloom's Taxonomy.JMIR dermatology · 2025Article
- Strategies for enhancing PHC accessibility through mobile and capsule clinics: a spatial location allocation study in China.BMC health services research · 2025Article
- ChatGPT Applications in Nursing: Current Status and Future Perspectives.Nursing open · 2025Review
- Developing artificial intelligence tools for institutional review board pre-review: A pilot study on ChatGPT's accuracy and reproducibility.PLOS digital health · 2025Article
- Customized GPT-4V(ision) for radiographic diagnosis: can large language model detect supernumerary teeth?BMC oral health · 2025Article
- Multiple large language models versus experienced physicians in diagnosing challenging cases with gastrointestinal symptoms.NPJ digital medicine · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundIn the evolving field of health care, multimodal generative artificial intelligence (AI) systems, such as ChatGPT-4 with vision (ChatGPT-4V), represent a significant advancement, as they integrate visual data with text data. This integration has the potential to revolutionize clinical diagnostics by offering more comprehensive analysis capabilities. However, the impact on diagnostic accuracy of using image data to augment ChatGPT-4 remains unclear.
objectiveThis study aims to assess the impact of adding image data on ChatGPT-4's diagnostic accuracy and provide insights into how image data integration can enhance the accuracy of multimodal AI in medical diagnostics. Specifically, this study endeavored to compare the diagnostic accuracy between ChatGPT-4V, which processed both text and image data, and its counterpart, ChatGPT-4, which only uses text data.
methodsWe identified a total of 557 case reports published in the American Journal of Case Reports from January 2022 to March 2023. After excluding cases that were nondiagnostic, pediatric, and lacking image data, we included 363 case descriptions with their final diagnoses and associated images. We compared the diagnostic accuracy of ChatGPT-4V and ChatGPT-4 without vision based on their ability to include the final diagnoses within differential diagnosis lists. Two independent physicians evaluated their accuracy, with a third resolving any discrepancies, ensuring a rigorous and objective analysis.
resultsThe integration of image data into ChatGPT-4V did not significantly enhance diagnostic accuracy, showing that final diagnoses were included in the top 10 differential diagnosis lists at a rate of 85.1% (n=309), comparable to the rate of 87.9% (n=319) for the text-only version (P=.33). Notably, ChatGPT-4V's performance in correctly identifying the top diagnosis was inferior, at 44.4% (n=161), compared with 55.9% (n=203) for the text-only version (P=.002, χ
conclusionsOur findings reveal that currently, ChatGPT-4V predominantly relies on textual data, limiting its ability to fully use the diagnostic potential of visual information. This study underscores the need for further development of multimodal generative AI systems to effectively integrate and use clinical image data. Enhancing the diagnostic performance of such AI systems through improved multimodal data integration could significantly benefit patient care by providing more accurate and comprehensive diagnostic insights. Future research should focus on overcoming these limitations, paving the way for the practical application of advanced AI in medicine.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.