ArticleJournal of the American College of Radiology : JACR2020
Multi-Institutional Assessment and Crowdsourcing Evaluation of Deep Learning for Automated Classification of Breast Density.
Article in Journal of the American College of Radiology : JACR, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 29 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
29 citing papers in PubMed.
- Leveraging deep learning to infer continuous predictions from ordinal labels in medical imaging.PLOS digital health · 2026Article
- Head-to-head comparisons of breast density assessment models using deep learning on digital and synthetic mammograms.Journal of medical imaging (Bellingham, Wash.) · 2026Article
- Using artificial intelligence system for assisting the classification of breast ultrasound glandular tissue components in dense breast tissue.Scientific reports · 2025Article
- Artificial intelligence and consistency in patient care: a large-scale longitudinal study of mammographic density assessment.BJR artificial intelligence · 2025Article
- The Challenge of Deep Learning for the Prevention and Automatic Diagnosis of Breast Cancer: A Systematic Review.Diagnostics (Basel, Switzerland) · 2024Review
- The top 100 most-cited articles on artificial intelligence in breast radiology: a bibliometric analysis.Insights into imaging · 2024Article
- Privacy preservation for federated learning in health care.Patterns (New York, N.Y.) · 2024Review
- Differential Diagnosis of OKC and SBC on Panoramic Radiographs: Leveraging Deep Learning Algorithms.Diagnostics (Basel, Switzerland) · 2024Article
- Artificial intelligence-based image analysis in clinical testing: lessons from cervical cancer screening.Journal of the National Cancer Institute · 2024Article
- Removing Radiographic Markers Using Deep Learning to Enable Image Sharing.Radiology. Artificial intelligence · 2023Article
- Mammographic Breast Density Model Using Semi-Supervised Learning Reduces Inter-/Intra-Reader Variability.Diagnostics (Basel, Switzerland) · 2023Article
- Article
- Collaborative Privacy-preserving Approaches for Distributed Deep Learning Using Multi-Institutional Data.Radiographics : a review publication of the Radiological Society of North America, Inc · 2023Article
- Evaluation of an Artificial Intelligence System for Retinopathy of Prematurity Screening in Nepal and Mongolia.Ophthalmology science · 2022Article
- ACR's Connect and AI-LAB technical framework.JAMIA open · 2022Article
- Improving the repeatability of deep learning models with Monte Carlo dropout.NPJ digital medicine · 2022Article
- How Machine Learning is Powering Neuroimaging to Improve Brain Health.Neuroinformatics · 2022Review
- Federated Learning for Multicenter Collaboration in Ophthalmology: Improving Classification Performance in Retinopathy of Prematurity.Ophthalmology. Retina · 2022Article
- Improved Training Efficiency for Retinopathy of Prematurity Deep Learning Models Using Comparison versus Class Labels.Ophthalmology science · 2022Article
- Mammographic Density Assessment by Artificial Intelligence-Based Computer-Assisted Diagnosis: A Comparison with Automated Volumetric Assessment.Journal of digital imaging · 2022Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
16 authors.
Funding
Abstract
objectiveWe developed deep learning algorithms to automatically assess BI-RADS breast density.
methodsUsing a large multi-institution patient cohort of 108,230 digital screening mammograms from the Digital Mammographic Imaging Screening Trial, we investigated the effect of data, model, and training parameters on overall model performance and provided crowdsourcing evaluation from the attendees of the ACR 2019 Annual Meeting.
resultsOur best-performing algorithm achieved good agreement with radiologists who were qualified interpreters of mammograms, with a four-class κ of 0.667. When training was performed with randomly sampled images from the data set versus sampling equal number of images from each density category, the model predictions were biased away from the low-prevalence categories such as extremely dense breasts. The net result was an increase in sensitivity and a decrease in specificity for predicting dense breasts for equal class compared with random sampling. We also found that the performance of the model degrades when we evaluate on digital mammography data formats that differ from the one that we trained on, emphasizing the importance of multi-institutional training sets. Lastly, we showed that crowdsourced annotations, including those from attendees who routinely read mammograms, had higher agreement with our algorithm than with the original interpreting radiologists.
conclusionWe demonstrated the possible parameters that can influence the performance of the model and how crowdsourcing can be used for evaluation. This study was performed in tandem with the development of the ACR AI-LAB, a platform for democratizing artificial intelligence.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.