Observational studyDiagnostic and interventional radiology (Ankara, Turkey)2025
Evaluating text and visual diagnostic capabilities of large language models on questions related to the Breast Imaging Reporting and Data System Atlas 5
Observational study in Diagnostic and interventional radiology (Ankara, Turkey), 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 15 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
15 citing papers in PubMed.
- From zero-shot to fine-tuning: optimize large language models for error detection of ultrasound reports.Insights into imaging · 2026Article
- Diagnostic accuracy and consistency of ChatGPT-4o in radiology: influence of image, clinical data, and answer options on performance.Diagnostic and interventional radiology (Ankara, Turkey) · 2026Article
- Large Language Model Translation of BI-RADS Breast Imaging Reports into Arabic: A Blinded Expert Evaluation of Diagnostic Communication Safety.Diagnostics (Basel, Switzerland) · 2026Article
- Evaluation of ChatGPT-4o in Breast Cancer Screening: Insights from the 5th Edition BI-RADS Atlas and ACR Guidelines.Journal of imaging informatics in medicine · 2026Article
- Report Generation System for Slit-Lamp Image Interpretation Using Vision-Language Models.Ophthalmology and therapy · 2026Article
- Comparative performance evaluation of ChatGPT-4 Omni and Gemini Advanced in the Turkish Dentistry Specialization Exam.BMC medical education · 2026Article
- Reply: evaluating text and visual diagnostic capabilities of large language models on questions related to the Breast Imaging Reporting and Data System (BI-RADS) Atlas 5Diagnostic and interventional radiology (Ankara, Turkey) · 2026Article
- Performance of ChatGPT-4o, Claude 3 Opus, and DeepSeek-R1 in BI-RADS Category 4 Classification and Malignancy Prediction From Mammography Reports: Retrospective Diagnostic Study.JMIR medical informatics · 2025Article
- Article
- Retrieval-augmented generation for answering Breast Imaging Reporting and Data System (BI-RADS)-related questions with large language models.Diagnostic and interventional radiology (Ankara, Turkey) · 2025Article
- Diagnostic accuracy of generative large language artificial intelligence models for the assessment of dental crowding.BMC oral health · 2025Article
- Large Language Models in Cancer Imaging: Applications and Future Perspectives.Journal of clinical medicine · 2025Review
- Diagnostic performance of multimodal large language models in radiological quiz cases: the effects of prompt engineering and input conditions.Ultrasonography (Seoul, Korea) · 2025Article
- A Novel Multimodal Large Language Model for Interpreting Image-Based Ophthalmology Case Questions: Comparative Analysis of Multiple-Choice and Open-Ended Response.Beyoglu eye journal · 2025Article
- Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
purposeThis study aimed to evaluate the performance of large language models (LLMs) and multimodal LLMs in interpreting the Breast Imaging Reporting and Data System (BI-RADS) categories and providing clinical management recommendations for breast radiology in text-based and visual questions.
methodsThis cross-sectional observational study involved two steps. In the first step, we compared ten LLMs (namely ChatGPT 4o, ChatGPT 4, ChatGPT 3.5, Google Gemini 1.5 Pro, Google Gemini 1.0, Microsoft Copilot, Perplexity, Claude 3.5 Sonnet, Claude 3 Opus, and Claude 3 Opus 200K), general radiologists, and a breast radiologist using 100 text-based multiple-choice questions (MCQs) related to the BI-RADS Atlas 5
resultsClaude 3.5 Sonnet achieved the highest accuracy in text-based MCQs (90%), followed by ChatGPT 4o (89%), outperforming all other LLMs and general radiologists (78% and 76%) (
conclusionAlthough LLMs such as Claude 3.5 Sonnet and ChatGPT 4o show promise in text-based BI-RADS assessments, their limitations in visual diagnostics suggest they should be used cautiously and under radiologists' supervision to avoid misdiagnoses. CLINICAL SIGNIFICANCE: This study demonstrates that while LLMs exhibit strong capabilities in text-based BI-RADS assessments, their visual diagnostic abilities are currently limited, necessitating further development and cautious application in clinical practice.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.