ArticleJournal of the American Medical Informatics Association : JAMIA2023
Inferring cancer disease response from radiology reports using large language models with data augmentation and prompting.
Article in Journal of the American Medical Informatics Association : JAMIA, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 23 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
23 citing papers in PubMed.
- Automated Clinical Information Extraction from Diagnostic and Nondiagnostic Radiology Reports Using Modern Language Models.Journal of imaging informatics in medicine · 2026Article
- Leveraging Large Language Models for Accurate AO Fracture Classification from CT Text Reports.Journal of imaging informatics in medicine · 2026Article
- Review
- Review
- Development and Validation of a Generative Artificial Intelligence-Based Pipeline for Automated Clinical Data Extraction From Electronic Health Records: Technical Implementation Study.JMIR bioinformatics and biotechnology · 2026Article
- Performance of ChatGPT-4o, Claude 3 Opus, and DeepSeek-R1 in BI-RADS Category 4 Classification and Malignancy Prediction From Mammography Reports: Retrospective Diagnostic Study.JMIR medical informatics · 2025Article
- Trends and Trajectories in the Rise of Large Language Models in Radiology: Scoping Review.JMIR medical informatics · 2025Article
- Artificial intelligence for immunotherapy response assessment in lung cancer using PET/CT reports.Japanese journal of radiology · 2025Article
- Impact of large language models and vision deep learning models in predicting neoadjuvant rectal score for rectal cancer treated with neoadjuvant chemoradiation.BMC medical imaging · 2025Article
- Empirical evaluation of artificial intelligence distillation techniques for ascertaining cancer outcomes from electronic health records.NPJ digital medicine · 2025Article
- Article
- Large Language Models in Cancer Imaging: Applications and Future Perspectives.Journal of clinical medicine · 2025Review
- Large Language Model Applications for Health Information Extraction in Oncology: Scoping Review.JMIR cancer · 2025Article
- A survey of NLP methods for oncology in the past decade with a focus on cancer registry applications.Artificial intelligence review · 2025Article
- Large language models in cancer: potentials, risks, and safeguards.BJR artificial intelligence · 2025Review
- Uncertainty-aware automatic TNM staging classification for [BMC medical informatics and decision making · 2024Article
- A Narrative Review on the Application of Large Language Models to Support Cancer Care and Research.Yearbook of medical informatics · 2024Review
- Year 2023 in Biomedical Natural Language Processing: a Tribute to Large Language Models and Generative AI.Yearbook of medical informatics · 2024Article
- Utilizing ChatGPT for Curriculum Learning in Developing a Clinical Grade Pneumothorax Detection Model: A Multisite Validation Study.Journal of clinical medicine · 2024Article
- Understanding enterprise data warehouses to support clinical and translational research: impact, sustainability, demand management, and accessibility.Journal of the American Medical Informatics Association : JAMIA · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
24 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectiveTo assess large language models on their ability to accurately infer cancer disease response from free-text radiology reports. MATERIALS AND
methodsWe assembled 10 602 computed tomography reports from cancer patients seen at a single institution. All reports were classified into: no evidence of disease, partial response, stable disease, or progressive disease. We applied transformer models, a bidirectional long short-term memory model, a convolutional neural network model, and conventional machine learning methods to this task. Data augmentation using sentence permutation with consistency loss as well as prompt-based fine-tuning were used on the best-performing models. Models were validated on a hold-out test set and an external validation set based on Response Evaluation Criteria in Solid Tumors (RECIST) classifications.
resultsThe best-performing model was the GatorTron transformer which achieved an accuracy of 0.8916 on the test set and 0.8919 on the RECIST validation set. Data augmentation further improved the accuracy to 0.8976. Prompt-based fine-tuning did not further improve accuracy but was able to reduce the number of training reports to 500 while still achieving good performance. DISCUSSION: These models could be used by researchers to derive progression-free survival in large datasets. It may also serve as a decision support tool by providing clinicians an automated second opinion of disease response.
conclusionsLarge clinical language models demonstrate potential to infer cancer disease response from radiology reports at scale. Data augmentation techniques are useful to further improve performance. Prompt-based fine-tuning can significantly reduce the size of the training dataset.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.