ArticleThe British journal of radiology2025
Assessing the performance of ChatGPT and Bard/Gemini against radiologists for Prostate Imaging-Reporting and Data System classification based on prostate multiparametric MRI text reports.
Article in The British journal of radiology, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 17 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
17 citing papers in PubMed.
- Exploring the feasibility of inferring prostate cancer pathological grade from multiparametric MRI text reports using natural language processing: assessment of four large language models.Abdominal radiology (New York) · 2026Article
- Large language models and prostate MRI reporting: a stringent testbed for safe deployment under evolving AI, health-data, and cybersecurity regulation.Abdominal radiology (New York) · 2026Review
- Large Language Models Versus Human Readers in CAD-RADS 2.0 Categorization of Coronary CT Angiography Reports.Journal of imaging informatics in medicine · 2026Article
- Can chatGPT-4o reliably standardize PSMA PET/CT and PET/MRI reports using PROMISE V2 criteria? - An exploratory study.EJNMMI research · 2026Article
- PI-RADS v2.1 for prostate MRI: current applications, ongoing debates, and synergies with advanced technologies-a narrative review.Translational andrology and urology · 2026Review
- Evaluation of large language models for VI-RADS reports: a comparative analysis of zero-shot and few-shot prompting.BMC medical imaging · 2026Article
- Retrieval-augmented generation-enhanced large language models for comprehensive CAD-RADS 2.0 categorization from structured coronary CTA reports.Radiologie (Heidelberg, Germany) · 2026Article
- [Research progress of large language models in tumor diagnosis: applications in textual reports and medical imaging].Nan fang yi ke da xue xue bao = Journal of Southern Medical University · 2026Review
- Evaluation of large language models in assigning PI-RADS v2.1 categories for prostate MRI reports.BMC urology · 2026Article
- Evaluation of large language models for PI-RADS score extraction from free-text prostate MRI reports: a comparative study with human readers.Frontiers in oncology · 2026Article
- Zero-Shot PI-RADS Version 2.1 Scoring with ChatGPT-4 Turbo and Llama 3: Diagnostic Performance and Agreement with Abdominal Radiologists.Radiology. Imaging cancer · 2026Article
- Precision oncology: from large language models to multi-agent systems.Frontiers in oncology · 2026Review
- Artificial intelligence for immunotherapy response assessment in lung cancer using PET/CT reports.Japanese journal of radiology · 2025Article
- In-context learning enables large language models to achieve human-level performance in spinal instability neoplastic score classification from synthetic CT and MRI reports.La Radiologia medica · 2025Article
- AI in Prostate MRI: A Task-Based Review.Journal of the Korean Society of Radiology · 2025Review
- Evaluation of large language models in generating pulmonary nodule follow-up recommendations.European journal of radiology open · 2025Article
- Large Language Models in Cancer Imaging: Applications and Future Perspectives.Journal of clinical medicine · 2025Review
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
Abstract
objectivesLarge language models (LLMs) have shown potential for clinical applications. This study assesses their ability to assign Prostate Imaging-Reporting and Data System (PI-RADS) categories based on clinical text reports.
methodsOne hundred consecutive biopsy-naïve patients' multiparametric prostate MRI reports were independently classified by 2 uroradiologists, ChatGPT-3.5 (GPT-3.5), ChatGPT-4o mini (GPT-4), Bard, and Gemini. Original report classifications were considered definitive.
resultsOut of 100 MRIs, 52 were originally reported as PI-RADS 1-2, 9 PI-RADS 3, 19 PI-RADS 4, and 20 PI-RADS 5. Radiologists demonstrated 95% and 90% accuracy, while GPT-3.5 and Bard both achieved 67%. Accuracy of the updated versions of LLMs increased to 83% (GTP-4) and 79% (Gemini), respectively. In low suspicion studies (PI-RADS 1-2), Bard and Gemini (F1: 0.94, 0.98, respectively) outperformed GPT-3.5 and GTP-4 (F1:0.77, 0.94, respectively), whereas for high probability MRIs (PI-RADS 4-5), GPT-3.5 and GTP-4 (F1: 0.95, 0.98, respectively) outperformed Bard and Gemini (F1: 0.71, 0.87, respectively). Bard assigned a non-existent PI-RADS 6 "hallucination" for 2 patients. Inter-reader agreements (Κ) between the original reports and the senior radiologist, junior radiologist, GPT-3.5, GTP-4, BARD, and Gemini were 0.93, 0.84, 0.65, 0.86, 0.57, and 0.81, respectively.
conclusionsRadiologists demonstrated high accuracy in PI-RADS classification based on text reports, while GPT-3.5 and Bard exhibited poor performance. GTP-4 and Gemini demonstrated improved performance compared to their predecessors. ADVANCES IN KNOWLEDGE: This study highlights the limitations of LLMs in accurately classifying PI-RADS categories from clinical text reports. While the performance of LLMs has improved with newer versions, caution is warranted before integrating such technologies into clinical practice.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.