Evidence map›Paper›PMID 33948701›Full record

ArticleEuropean radiology2021

Impact of artificial intelligence support on accuracy and reading time in breast tomosynthesis image interpretation: a multi-reader multi-case study.

Suzanne L van Winkel, Alejandro Rodríguez-Ruiz, Linda Appelman, Albert Gubern-Mérida, Nico Karssemeijer, Jonas Teuwen, Alexander J T Wanders, Ioannis Sechopoulos, Ritse M Mann

Registry-linked trialOpen access · hybridAbstract read
In one paragraph

Article in European radiology, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT05048095 (The Use of AI as a Third Reader and During Consensus in a Double Reading Breast Cancer Screening Program in Sweden), which is not on this map. Cited by 50 papers, 3 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
50citing papers in PubMed, 3 pooled it
8.1field-weighted citation impact, top 2% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

NCT05048095 completednot on this map

The Use of AI as a Third Reader and During Consensus in a Double Reading Breast Cancer Screening Program in Sweden

TypeobservationalSponsorOstergotland County Council, SwedenRan2021 to 2022Enrolled15,500ConditionsBreast CancerArmsAI cancer detection system
3 · Its place in the literature

Who cites it

50 citing papers in PubMed, 3 syntheses or guidelines pooled it, 91 citations in OpenAlex.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Review
  12. Article
  13. Article
  14. Impact of artificial intelligence assistance on bone scintigraphy diagnosis.Physical and engineering sciences in medicine · 2025
    Article
  15. Article
  16. Article
  17. Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

9 authors at 2 institutions in 2 countries.

Suzanne L van WinkelDepartment of Medical Imaging, Radboud University Medical Center, PO Box 9101, 6500 HB Nijmegen, Geert Grooteplein 10, 6525 GA, Post 766, Nijmegen, The Netherlands. Suzanne.vanWinkel@radboudumc.nl.ORCID http://orcid.org/0000-0001-7273-4386
Alejandro Rodríguez-RuizScreenPoint Medical BV, Toernooiveld 300, 6525 EC, Nijmegen, The Netherlands.
Linda AppelmanDepartment of Medical Imaging, Radboud University Medical Center, PO Box 9101, 6500 HB Nijmegen, Geert Grooteplein 10, 6525 GA, Post 766, Nijmegen, The Netherlands.
Albert Gubern-MéridaScreenPoint Medical BV, Toernooiveld 300, 6525 EC, Nijmegen, The Netherlands.
Nico KarssemeijerDepartment of Medical Imaging, Radboud University Medical Center, PO Box 9101, 6500 HB Nijmegen, Geert Grooteplein 10, 6525 GA, Post 766, Nijmegen, The Netherlands.
Jonas TeuwenDepartment of Medical Imaging, Radboud University Medical Center, PO Box 9101, 6500 HB Nijmegen, Geert Grooteplein 10, 6525 GA, Post 766, Nijmegen, The Netherlands.
Alexander J T WandersBevolkingsonderzoek Zuid-West Borstkanker, Laan 20, 2512 GB, Den Haag, The Netherlands.
Ioannis SechopoulosDepartment of Medical Imaging, Radboud University Medical Center, PO Box 9101, 6500 HB Nijmegen, Geert Grooteplein 10, 6525 GA, Post 766, Nijmegen, The Netherlands.
Ritse M MannDepartment of Medical Imaging, Radboud University Medical Center, PO Box 9101, 6500 HB Nijmegen, Geert Grooteplein 10, 6525 GA, Post 766, Nijmegen, The Netherlands. Ritse.Mann@radboudumc.nl.
Radboud University Nijmegen · NLBorstkanker Onderzoek Groep · NL

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

objectivesDigital breast tomosynthesis (DBT) increases sensitivity of mammography and is increasingly implemented in breast cancer screening. However, the large volume of images increases the risk of reading errors and reading time. This study aims to investigate whether the accuracy of breast radiologists reading wide-angle DBT increases with the aid of an artificial intelligence (AI) support system. Also, the impact on reading time was assessed and the stand-alone performance of the AI system in the detection of malignancies was compared to the average radiologist.

methodsA multi-reader multi-case study was performed with 240 bilateral DBT exams (71 breasts with cancer lesions, 70 breasts with benign findings, 339 normal breasts). Exams were interpreted by 18 radiologists, with and without AI support, providing cancer suspicion scores per breast. Using AI support, radiologists were shown examination-based and region-based cancer likelihood scores. Area under the receiver operating characteristic curve (AUC) and reading time per exam were compared between reading conditions using mixed-models analysis of variance.

resultsOn average, the AUC was higher using AI support (0.863 vs 0.833; p = 0.0025). Using AI support, reading time per DBT exam was reduced (p < 0.001) from 41 (95% CI = 39-42 s) to 36 s (95% CI = 35- 37 s). The AUC of the stand-alone AI system was non-inferior to the AUC of the average radiologist (+0.007, p = 0.8115).

conclusionsRadiologists improved their cancer detection and reduced reading time when evaluating DBT examinations using an AI reading support system. KEY POINTS: • Radiologists improved their cancer detection accuracy in digital breast tomosynthesis (DBT) when using an AI system for support, while simultaneously reducing reading time. • The stand-alone breast cancer detection performance of an AI system is non-inferior to the average performance of radiologists for reading digital breast tomosynthesis exams. • The use of an AI support system could make advanced and more reliable imaging techniques more accessible and could allow for more cost-effective breast screening programs with DBT.

Indexed as

Artificial IntelligenceBreast NeoplasmsBreastEarly Detection of CancerFemaleHumansMammographyArtificial intelligence (AI)Breast cancerDigital breast tomosynthesis (DBT)MammographyMass screening

Identifiers

PMID33948701
PMCPMC8523448
OpenAlexW3159635797

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.