Evidence map›Paper›PMID 30993432›Full record

ArticleEuropean radiology2019

Can we reduce the workload of mammographic screening by automatic identification of normal exams with artificial intelligence? A feasibility study.

Alejandro Rodriguez-Ruiz, Kristina Lång, Albert Gubern-Merida, Jonas Teuwen, Mireille Broeders, Gisella Gennaro, Paola Clauser, Thomas H Helbich, Margarita Chevalier, Thomas Mertelmeier and 5 more

2 registry-linked trialsOpen access · hybridAbstract read
In one paragraph

Article in European radiology, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to 2 registered trials, which are not on this map. Cited by 97 papers, 4 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
97citing papers in PubMed, 4 pooled it
13.5field-weighted citation impact, top 1% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

NCT04949776 nacompletednot on this mapstarted 2022, after this paper: background citation

New Strategies Based on Artificial Intelligence in Breast Cancer Screening Programs in Córdoba With Digital Mammography and Digital Breast Tomosynthesis. A Prospective Evaluation.

TypeinterventionalSponsorMaimónides Biomedical Research Institute of CórdobaRan2022 to 2024Enrolled31,301ConditionsBreast Cancer, ScreeningArmsMammograms
NCT05048095 completednot on this mapstarted 2021, after this paper: background citation

The Use of AI as a Third Reader and During Consensus in a Double Reading Breast Cancer Screening Program in Sweden

TypeobservationalSponsorOstergotland County Council, SwedenRan2021 to 2022Enrolled15,500ConditionsBreast CancerArmsAI cancer detection system
3 · Its place in the literature

Who cites it

97 citing papers in PubMed, 4 syntheses or guidelines pooled it, 210 citations in OpenAlex.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Article
  6. Article
  7. Article
  8. Review
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Review
  17. Article
  18. Article
  19. Article
  20. Review

37 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

15 authors at 10 institutions in 8 countries.

Alejandro Rodriguez-RuizDepartment of Radiology and Nuclear Medicine, Radboud University Medical Center, PO Box 9101, 6500 HB, Nijmegen, The Netherlands.
Kristina LångInstitute for Biomedical Engineering, ETH Zurich, Gloriastrasse 35, 8092, Zürich, Switzerland.
Albert Gubern-MeridaScreenPoint Medical BV, Stationplein 26, 6512 AB, Nijmegen, The Netherlands.
Jonas TeuwenDepartment of Radiology and Nuclear Medicine, Radboud University Medical Center, PO Box 9101, 6500 HB, Nijmegen, The Netherlands.
Mireille BroedersDepartment for Health Evidence, Radboud University Medical Center, P.O. Box 9101, 6500 HB, Nijmegen, The Netherlands.
Gisella GennaroVeneto Institute of Oncology (IOV)-IRCCS, via Gattamelata 64, 35128, Padua, Italy.
Paola ClauserDepartment of Biomedical Imaging and Image-Guided Therapy, Division of Molecular and Gender Imaging, Medical University of Vienna, Waehringer Guertel 18-20, 1090, Vienna, Austria.
Thomas H HelbichDepartment of Biomedical Imaging and Image-Guided Therapy, Division of Molecular and Gender Imaging, Medical University of Vienna, Waehringer Guertel 18-20, 1090, Vienna, Austria.
Margarita ChevalierMedical Physics Group, Radiology Department, Faculty of Medicine, Universidad Complutense de Madrid, Pza. Ramón y Cajal s/n, 28040, Madrid, Spain.
Thomas MertelmeierSiemens Healthcare GmbH, Diagnostic Imaging, X-Ray Products, Technology & Concepts, Siemensstr. 3, 91301, Forchheim, Germany.
Matthew G WallisCambridge Breast Unit and NIHR Biomedical Research Unit, Box 97, Cambridge University Hospitals NHS Foundation Trust, Cambridge Biomedical Campus, Hills Road, Cambridge, CB2 0QQ, UK.
Ingvar AnderssonUnilabs Breast Center, Skåne University Hospital, Jan Waldenströms gata 22, SE-20502, Malmö, Sweden.
Sophia ZackrissonDiagnostic Radiology, Department of Translational Medicine, Lund University, Skåne University Hospital, SE-20502, Malmö, Sweden.
Ioannis SechopoulosDepartment of Radiology and Nuclear Medicine, Radboud University Medical Center, PO Box 9101, 6500 HB, Nijmegen, The Netherlands.
Ritse M MannDepartment of Radiology and Nuclear Medicine, Radboud University Medical Center, PO Box 9101, 6500 HB, Nijmegen, The Netherlands. Ritse.Mann@radboudumc.nl.
Medical University of Vienna · ATRadboud University Medical Center · NLRadboud University Nijmegen · NLSkåne University Hospital · SECambridge University Hospitals NHS Foundation Trust · GBDutch Expert Centre for Screening · NLInstitute for Biomedical Engineering · CHIstituti di Ricovero e Cura a Carattere Scientifico · ITSiemens Healthcare (Germany) · DEUniversidad Complutense de Madrid · ES

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

purposeTo study the feasibility of automatically identifying normal digital mammography (DM) exams with artificial intelligence (AI) to reduce the breast cancer screening reading workload. METHODS AND MATERIALS: A total of 2652 DM exams (653 cancer) and interpretations by 101 radiologists were gathered from nine previously performed multi-reader multi-case receiver operating characteristic (MRMC ROC) studies. An AI system was used to obtain a score between 1 and 10 for each exam, representing the likelihood of cancer present. Using all AI scores between 1 and 9 as possible thresholds, the exams were divided into groups of low- and high likelihood of cancer present. It was assumed that, under the pre-selection scenario, only the high-likelihood group would be read by radiologists, while all low-likelihood exams would be reported as normal. The area under the reader-averaged ROC curve (AUC) was calculated for the original evaluations and for the pre-selection scenarios and compared using a non-inferiority hypothesis.

resultsSetting the low/high-likelihood threshold at an AI score of 5 (high likelihood > 5) results in a trade-off of approximately halving (- 47%) the workload to be read by radiologists while excluding 7% of true-positive exams. Using an AI score of 2 as threshold yields a workload reduction of 17% while only excluding 1% of true-positive exams. Pre-selection did not change the average AUC of radiologists (inferior 95% CI > - 0.05) for any threshold except at the extreme AI score of 9.

conclusionIt is possible to automatically pre-select exams using AI to significantly reduce the breast cancer screening reading workload. KEY POINTS: • There is potential to use artificial intelligence to automatically reduce the breast cancer screening reading workload by excluding exams with a low likelihood of cancer. • The exclusion of exams with the lowest likelihood of cancer in screening might not change radiologists' breast cancer detection performance. • When excluding exams with the lowest likelihood of cancer, the decrease in true-positive recalls would be balanced by a simultaneous reduction in false-positive recalls.

Indexed as

Artificial IntelligenceBreast NeoplasmsEarly Detection of CancerFalse Negative ReactionsFalse Positive ReactionsFeasibility StudiesFemaleHumansMammographyMass ScreeningProbabilityRadiologistsROC CurveWorkloadArtificial intelligenceBreast cancerDeep learningMammographyScreening

Identifiers

PMID30993432
PMCPMC6682851
OpenAlexW2936218643

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.