ArticleJapanese journal of radiology2024
GPT-4 Turbo with Vision fails to outperform text-only GPT-4 Turbo in the Japan Diagnostic Radiology Board Examination.
Article in Japanese journal of radiology, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 20 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
20 citing papers in PubMed.
- Performance of large language models on the Japanese cardiovascular surgery board examination: a comparative analysis of eight contemporary AI models with educational implications.General thoracic and cardiovascular surgery · 2026Article
- Performance of Reasoning and Comparator Large Language Models on Nephrology Multiple-choice Questions.JMA journal · 2026Article
- AI achieves board-level performance on the Japan diagnostic radiology board examination through direct image interpretation.Japanese journal of radiology · 2026Article
- Context-Aware Sentence Classification of Radiology Reports Using Synthetic Data: Development and Validation Study.Journal of medical Internet research · 2026Article
- Intra-axial primary brain tumor differentiation: comparing large language models on structured MRI reports vs. radiologists on images.European radiology · 2026Article
- Article
- Assessing accuracy and legitimacy of multimodal large language models on Japan Diagnostic Radiology Board Examination.Japanese journal of radiology · 2026Article
- Performance of multimodal large language models in the Japanese surgical specialist examination.BMC medical education · 2025Article
- Article
- Role of Artificial Intelligence in Surgical Training by Assessing GPT-4 and GPT-4o on the Japan Surgical Board Examination With Text-Only and Image-Accompanied Questions: Performance Evaluation Study.JMIR medical education · 2025Article
- Article
- The critical need for an open medical imaging database in Japan: implications for global health and AI development.Japanese journal of radiology · 2025Article
- Evaluating ChatGPT-4's Performance in Identifying Radiological Anatomy in FRCR Part 1 Examination Questions.The Indian journal of radiology & imaging · 2025Article
- Evaluating AI proficiency in nuclear cardiology: Large language models take on the board preparation exam.Journal of nuclear cardiology : official publication of the American Society of Nuclear Cardiology · 2025Article
- It is Not Time to Kick Out Radiologists.Asian bioethics review · 2025Article
- Diagnostic accuracy of vision-language models on Japanese diagnostic radiology, nuclear medicine, and interventional radiology specialty board examinations.Japanese journal of radiology · 2024Article
- Diagnostic performances of GPT-4o, Claude 3 Opus, and Gemini 1.5 Pro in "Diagnosis Please" cases.Japanese journal of radiology · 2024Article
- Generative AI and large language models in nuclear medicine: current status and future prospects.Annals of nuclear medicine · 2024Review
- Article
- Evaluating AI Proficiency in Nuclear Cardiology: Large Language Models take on the Board Preparation Exam.medRxiv : the preprint server for health sciences · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
9 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
purposeTo assess the performance of GPT-4 Turbo with Vision (GPT-4TV), OpenAI's latest multimodal large language model, by comparing its ability to process both text and image inputs with that of the text-only GPT-4 Turbo (GPT-4 T) in the context of the Japan Diagnostic Radiology Board Examination (JDRBE). MATERIALS AND
methodsThe dataset comprised questions from JDRBE 2021 and 2023. A total of six board-certified diagnostic radiologists discussed the questions and provided ground-truth answers by consulting relevant literature as necessary. The following questions were excluded: those lacking associated images, those with no unanimous agreement on answers, and those including images rejected by the OpenAI application programming interface. The inputs for GPT-4TV included both text and images, whereas those for GPT-4 T were entirely text. Both models were deployed on the dataset, and their performance was compared using McNemar's exact test. The radiological credibility of the responses was assessed by two diagnostic radiologists through the assignment of legitimacy scores on a five-point Likert scale. These scores were subsequently used to compare model performance using Wilcoxon's signed-rank test.
resultsThe dataset comprised 139 questions. GPT-4TV correctly answered 62 questions (45%), whereas GPT-4 T correctly answered 57 questions (41%). A statistical analysis found no significant performance difference between the two models (P = 0.44). The GPT-4TV responses received significantly lower legitimacy scores from both radiologists than the GPT-4 T responses.
conclusionNo significant enhancement in accuracy was observed when using GPT-4TV with image input compared with that of using text-only GPT-4 T for JDRBE questions.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.