Evidence map›Paper›PMID 38476957›Full record

ReviewBJR artificial intelligence2024

Artificial intelligence in medicine: mitigating risks and maximizing benefits via quality assurance, quality control, and acceptance testing.

Usman Mahmood, Amita Shukla-Dave, Heang-Ping Chan, Karen Drukker, Ravi K Samala, Quan Chen, Daniel Vergara, Hayit Greenspan, Nicholas Petrick, Berkman Sahiner and 15 more

Open access · diamondAbstract readReview
In one paragraph

Review in BJR artificial intelligence, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 23 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
23citing papers in PubMed, 2 pooled it
3.8field-weighted citation impact, top 6% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

23 citing papers in PubMed, 2 syntheses or guidelines pooled it, 32 citations in OpenAlex.

  1. Pooled it
  2. Pooled it
  3. Trial
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Review
  11. Review
  12. Article
  13. Article
  14. Article
  15. Review
  16. Review
  17. Review
  18. Review
  19. Review
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

25 authors at 17 institutions in 5 countries.

Usman MahmoodDepartment of Medical Physics, Memorial Sloan-Kettering Cancer Center, New York, NY, 10065, United States.ORCID https://orcid.org/0000-0002-5170-5794
Amita Shukla-DaveDepartment of Medical Physics, Memorial Sloan-Kettering Cancer Center, New York, NY, 10065, United States.
Heang-Ping ChanDepartment of Radiology, University of Michigan, Ann Arbor, MI, 48109, United States.ORCID https://orcid.org/0000-0001-7777-9006
Karen DrukkerDepartment of Radiology, University of Chicago, Chicago, IL, 60637, United States.ORCID https://orcid.org/0000-0001-6544-3476
Ravi K SamalaOffice of Science and Engineering Laboratories, Center for Devices and Radiological Health, U.S. Food and Drug Administration, Silver Spring, MD, 20993, United States.ORCID https://orcid.org/0000-0002-6661-4801
Quan ChenDepartment of Radiation Oncology, Mayo Clinic Arizona, Phoenix, AZ, 85054, United States.
Daniel VergaraDepartment of Radiology, University of Washington, Seattle, WA, 98195, United States.ORCID https://orcid.org/0000-0003-0809-8308
Hayit GreenspanBiomedical Engineering and Imaging Institute, Department of Radiology, Icahn School of Medicine at Mt Sinai, New York, NY, 10029, United States.
Nicholas PetrickOffice of Science and Engineering Laboratories, Center for Devices and Radiological Health, U.S. Food and Drug Administration, Silver Spring, MD, 20993, United States.ORCID https://orcid.org/0000-0001-5167-8899
Berkman SahinerOffice of Science and Engineering Laboratories, Center for Devices and Radiological Health, U.S. Food and Drug Administration, Silver Spring, MD, 20993, United States.
Zhimin HuoTencent America, Palo Alto, CA, 94306, United States.
Ronald M SummersRadiology and Imaging Sciences, National Institutes of Health Clinical Center, Bethesda, MD, 20892, United States.ORCID https://orcid.org/0000-0001-8081-7376
Kenny H ChaOffice of Science and Engineering Laboratories, Center for Devices and Radiological Health, U.S. Food and Drug Administration, Silver Spring, MD, 20993, United States.
Georgia TourassiComputing and Computational Sciences Directorate, Oak Ridge National Laboratory, Oak Ridge, TN, 37830, United States.
Thomas M DesernoPeter L. Reichertz Institute for Medical Informatics, TU Braunschweig and Hannover Medical School, Braunschweig, Niedersachsen, 38106, Germany.ORCID https://orcid.org/0000-0003-3492-4407
Kevin T GrizzardDepartment of Radiology and Biomedical Imaging, Yale University School of Medicine, New Haven, CT, 06510, United States.
Janne J Näppi3D Imaging Research, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, 02114, United States.
Hiroyuki Yoshida3D Imaging Research, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, 02114, United States.
Daniele ReggeRadiology Unit, Candiolo Cancer Institute, FPO-IRCCS, Candiolo, 10060, Italy.
Richard MazurchukDivision of Cancer Prevention, National Cancer Institute, National Institutes of Health, Bethesda, MD, 20892, United States.
Kenji SuzukiInstitute of Innovative Research, Tokyo Institute of Technology, Midori-ku, Yokohama, Kanagawa, 226-8503, Japan.
Lia MorraDepartment of Control and Computer Engineering, Politecnico di Torino, Torino, Piemonte, 10129, Italy.
Henkjan HuismanRadboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, Gelderland, 6525 GA, Netherlands.
Samuel G ArmatoDepartment of Radiology, University of Chicago, Chicago, IL, 60637, United States.
Lubomir HadjiiskiDepartment of Radiology, University of Michigan, Ann Arbor, MI, 48109, United States.
United States Food and Drug Administration · USHarvard University · USMemorial Sloan Kettering Cancer Center · USUniversity of Chicago · USUniversity of Michigan · USKLA (United States) · USMayo Clinic Hospital · USMedizinische Hochschule Hannover · DENational Institutes of Health · USNational Institutes of Health Clinical Center · USOak Ridge National Laboratory · USPolitecnico di Torino · ITRadboud University Nijmegen · NLTokyo Institute of Technology · JPUniversity of Pisa · ITUniversity of Washington · USYale University · US

Funding

MEDICAL IMAGING AND DATA RESOURCE CENTER (MIDRC) YEAR 375N92020D00021 · NIBIB · UNIVERSITY OF CHICAGO · PI GIGER, MARYELLEN · 2021 to 2025
$16.6M
Biomarker-Based Tools for Treatment Response Decision Support of Bladder CancerU01CA232931 · NCI · UNIVERSITY OF MICHIGAN AT ANN ARBOR · PI ALVA, AJJAI SHIVARAM, HADJIYSKI, LUBOMIR M · 2019 to 2024
$3.1M
Advanced breast tomosynthesis reconstruction for improved cancer diagnosisR01CA214981 · NCI · UNIVERSITY OF MICHIGAN AT ANN ARBOR · PI CHAN, HEANG-PING, FESSLER, JEFFREY A · 2018 to 2022
$2.3M
Survival prediction in patients with progressive fibrosing interstitial lung diseaseR01HL164697 · NHLBI · MASSACHUSETTS GENERAL HOSPITAL · PI YOSHIDA, HIROYUKI · 2022 to 2025
$1.7M
Spectral precision imaging for early diagnosis of colorectal lesions with CT colonographyR01CA212382 · NCI · MASSACHUSETTS GENERAL HOSPITAL · PI YOSHIDA, HIROYUKI · 2018 to 2022
$1.5M
Computer Aided Detection for Radiologic ImagesZIACL040004 · CLC · CLINICAL CENTER · PI SUMMERS, RONALD M. · 2009 to 2025
$0k
NCI NIH HHS R01 CA212382NCI NIH HHS R01 CA214981NCI NIH HHS U01 CA232931NHLBI NIH HHS 75N92020D00021NHLBI NIH HHS R01 HL164697
6 · The paper itself

Abstract

The adoption of artificial intelligence (AI) tools in medicine poses challenges to existing clinical workflows. This commentary discusses the necessity of context-specific quality assurance (QA), emphasizing the need for robust QA measures with quality control (QC) procedures that encompass (1) acceptance testing (AT) before clinical use, (2) continuous QC monitoring, and (3) adequate user training. The discussion also covers essential components of AT and QA, illustrated with real-world examples. We also highlight what we see as the shared responsibility of manufacturers or vendors, regulators, healthcare systems, medical physicists, and clinicians to enact appropriate testing and oversight to ensure a safe and equitable transformation of medicine through AI.

Indexed as

acceptance testingartificial intelligencedeep learningmachine learningquality assurancequality controlradiology

Identifiers

PMID38476957
PMCPMC10928809
OpenAlexW4392405140

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.