ArticleEuropean radiology2024
Evaluating the performance of Generative Pre-trained Transformer-4 (GPT-4) in standardizing radiology reports.
Article in European radiology, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 40 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
40 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Effectiveness, Safety, and Workflow Burden of Large Language Model-Based Medical Report Generation: Systematic Review.Journal of medical Internet research · 2026Pooled it
- Simplifying radiology reports with large language models: privacy-compliant open- versus closed-weight models.European radiology · 2026Article
- Etiologic classification of suspected MINOCA using cardiovascular magnetic resonance reports: a comparison of a large language model and human readers.The international journal of cardiovascular imaging · 2026Article
- Zero-shot multimodal large language models underperform a domain-trained CNN baseline in pediatric wrist fracture detection.Scientific reports · 2026Article
- Adaptive Fast-Slow Large Language Model Framework for Multidimensional Classification of Prenatal Ultrasound Reports: Comparative Study.Journal of medical Internet research · 2026Article
- Article
- A robust vision language model for molecular status prediction and radiology report generation in adult-type diffuse gliomas.NPJ digital medicine · 2026Article
- Cross-Institutional Evaluation of Large Language Models for Radiology Diagnosis Extraction: A Prompt-Engineering Perspective.Journal of imaging informatics in medicine · 2026Article
- The Role of AI-Generated Clinical Image Descriptions in Enhancing Teledermatology Diagnosis: A Cross-Sectional Exploratory Study.Diagnostics (Basel, Switzerland) · 2026Article
- Limitations in Chest X-Ray Interpretation by Vision-Capable Large Language Models, Gemini 1.0, Gemini 1.5 Pro, GPT-4 Turbo, and GPT-4o.Diagnostics (Basel, Switzerland) · 2026Article
- Large Language Models for Psychiatric Diagnosis Based on Multicenter Real-World Clinical Records: Comparative Study.JMIR medical informatics · 2026Article
- Case-matched retrieval improves textual alignment of LLM-generated radiology impressions.PloS one · 2026Article
- The role of artificial intelligence in sports training: opportunities, challenges and future applications for competitive swimming.Biology of sport · 2026Article
- Trends and Trajectories in the Rise of Large Language Models in Radiology: Scoping Review.JMIR medical informatics · 2025Article
- Evaluating Large Language Models in Interpreting MRI Reports and Recommending Treatment for Vestibular Schwannoma.Diagnostics (Basel, Switzerland) · 2025Article
- One Step Closer to Conversational Medical Records: ChatGPT Parses Psoriasis Treatments from EMRs.Journal of clinical medicine · 2025Article
- State of the art review of AI in renal imaging.Abdominal radiology (New York) · 2025Review
- The Use of ChatGPT-4.0 to Simplify Breast Pathology Reports: A Study on Readability and Accuracy.Annals of surgical oncology · 2025Article
- Evaluating prompt and data perturbation sensitivity in large language models for radiology reports classification.JAMIA open · 2025Article
- Synthetic data trained open-source language models are feasible alternatives to proprietary models for radiology reporting.NPJ digital medicine · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
10 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectiveRadiology reporting is an essential component of clinical diagnosis and decision-making. With the advent of advanced artificial intelligence (AI) models like GPT-4 (Generative Pre-trained Transformer 4), there is growing interest in evaluating their potential for optimizing or generating radiology reports. This study aimed to compare the quality and content of radiologist-generated and GPT-4 AI-generated radiology reports.
methodsA comparative study design was employed in the study, where a total of 100 anonymized radiology reports were randomly selected and analyzed. Each report was processed by GPT-4, resulting in the generation of a corresponding AI-generated report. Quantitative and qualitative analysis techniques were utilized to assess similarities and differences between the two sets of reports.
resultsThe AI-generated reports showed comparable quality to radiologist-generated reports in most categories. Significant differences were observed in clarity (p = 0.027), ease of understanding (p = 0.023), and structure (p = 0.050), favoring the AI-generated reports. AI-generated reports were more concise, with 34.53 fewer words and 174.22 fewer characters on average, but had greater variability in sentence length. Content similarity was high, with an average Cosine Similarity of 0.85, Sequence Matcher Similarity of 0.52, BLEU Score of 0.5008, and BERTScore F1 of 0.8775.
conclusionThe results of this proof-of-concept study suggest that GPT-4 can be a reliable tool for generating standardized radiology reports, offering potential benefits such as improved efficiency, better communication, and simplified data extraction and analysis. However, limitations and ethical implications must be addressed to ensure the safe and effective implementation of this technology in clinical practice. CLINICAL RELEVANCE STATEMENT: The findings of this study suggest that GPT-4 (Generative Pre-trained Transformer 4), an advanced AI model, has the potential to significantly contribute to the standardization and optimization of radiology reporting, offering improved efficiency and communication in clinical practice. KEY POINTS: • Large language model-generated radiology reports exhibited high content similarity and moderate structural resemblance to radiologist-generated reports. • Performance metrics highlighted the strong matching of word selection and order, as well as high semantic similarity between AI and radiologist-generated reports. • Large language model demonstrated potential for generating standardized radiology reports, improving efficiency and communication in clinical settings.
Indexed as
Identifiers
37938381What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.