Evidence map›Paper›PMID 34089072›Full record

ArticleEuropean radiology2021

AI-based improvement in lung cancer detection on chest radiographs: results of a multi-reader study in NLST dataset.

Hyunsuk Yoo, Sang Hyup Lee, Chiara Daniela Arru, Ruhani Doda Khera, Ramandeep Singh, Sean Siebert, Dohoon Kim, Yuna Lee, Ju Hyun Park, Hye Joung Eom and 2 more

Registry-linked trialAbstract read
PubMed Publisher
In one paragraph

Article in European radiology, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT05489471 (A Prospective Study to Assess the Impact of an Artificial Intelligence System on Reporting of Chest X-rays, Evaluate the Ability of AI Driven Worklists to Improve Reporting Times and Improve Same Day CT Pathway for Suspected Lung Cancer), which is not on this map. Cited by 38 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
38citing papers in PubMed, 1 pooled it
8.4field-weighted citation impact, top 2% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

NCT05489471 unknown statusnot on this mapstarted 2023, after this paper: background citation

A Prospective Study to Assess the Impact of an Artificial Intelligence System on Reporting of Chest X-rays, Evaluate the Ability of AI Driven Worklists to Improve Reporting Times and Improve Same Day CT Pathway for Suspected Lung Cancer

TypeobservationalSponsorHull University Teaching Hospitals NHS TrustRan2023 to 2023Enrolled20,000ConditionsArtificial IntelligenceArmsArtificial intelligence review
3 · Its place in the literature

Who cites it

38 citing papers in PubMed, 1 synthesis or guideline pooled it, 81 citations in OpenAlex.

  1. Pooled it
  2. Article
  3. Review
  4. Article
  5. Article
  6. Article
  7. Article
  8. Review
  9. Review
  10. Review
  11. Review
  12. Article
  13. Article
  14. Review
  15. Article
  16. Article
  17. Article
  18. Article
  19. Review
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

12 authors at 4 institutions in 2 countries.

Hyunsuk YooLunit, Seoul, Korea.
Sang Hyup LeeLunit, Seoul, Korea.
Chiara Daniela ArruDivision of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital, 75 Blossom Court, Boston, MA, 02114, USA.
Ruhani Doda KheraDivision of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital, 75 Blossom Court, Boston, MA, 02114, USA.
Ramandeep SinghDivision of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital, 75 Blossom Court, Boston, MA, 02114, USA.
Sean SiebertDivision of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital, 75 Blossom Court, Boston, MA, 02114, USA.
Dohoon KimDepartment of Radiology, Seoul National University College of Medicine, Seoul, Korea.
Yuna LeeDepartment of Radiology, Seoul National University College of Medicine, Seoul, Korea.
Ju Hyun ParkSuwon Total Healthcare Center, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Youngin-si, Gyeongi-do, 16954, Korea.
Hye Joung EomCheju Halla General Hospital, 65 Doryeong-ro, Yeon-dong, Jeju-si, Jeju-do, Korea.
Subba R DigumarthyDivision of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital, 75 Blossom Court, Boston, MA, 02114, USA.
Mannudeep K KalraDivision of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital, 75 Blossom Court, Boston, MA, 02114, USA. mkalra@mgh.harvard.edu.ORCID http://orcid.org/0000-0001-9938-7476
Harvard University · USSeoul National University · KRCheju Halla General Hospital · KRKangbuk Samsung Hospital · KR

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

objectiveAssess if deep learning-based artificial intelligence (AI) algorithm improves reader performance for lung cancer detection on chest X-rays (CXRs).

methodsThis reader study included 173 images from cancer-positive patients (n = 98) and 346 images from cancer-negative patients (n = 196) selected from National Lung Screening Trial (NLST). Eight readers, including three radiology residents, and five board-certified radiologists, participated in the observer performance test. AI algorithm provided image-level probability of pulmonary nodule or mass on CXRs and a heatmap of detected lesions. Reader performance was compared with AUC, sensitivity, specificity, false-positives per image (FPPI), and rates of chest CT recommendations.

resultsWith AI, the average sensitivity of readers for the detection of visible lung cancer increased for residents, but was similar for radiologists compared to that without AI (0.61 [95% CI, 0.55-0.67] vs. 0.72 [95% CI, 0.66-0.77], p = 0.016 for residents, and 0.76 [95% CI, 0.72-0.81] vs. 0.76 [95% CI, 0.72-0.81, p = 1.00 for radiologists), while false-positive findings per image (FPPI) was similar for residents, but decreased for radiologists (0.15 [95% CI, 0.11-0.18] vs. 0.12 [95% CI, 0.09-0.16], p = 0.13 for residents, and 0.24 [95% CI, 0.20-0.29] vs. 0.17 [95% CI, 0.13-0.20], p < 0.001 for radiologists). With AI, the average rate of chest CT recommendation in patients positive for visible cancer increased for residents, but was similar for radiologists (54.7% [95% CI, 48.2-61.2%] vs. 70.2% [95% CI, 64.2-76.2%], p < 0.001 for residents and 72.5% [95% CI, 68.0-77.1%] vs. 73.9% [95% CI, 69.4-78.3%], p = 0.68 for radiologists), while that in cancer-negative patients was similar for residents, but decreased for radiologists (11.2% [95% CI, 9.6-13.1%] vs. 9.8% [95% CI, 8.0-11.6%], p = 0.32 for residents and 16.4% [95% CI, 14.7-18.2%] vs. 11.7% [95% CI, 10.2-13.3%], p < 0.001 for radiologists).

conclusionsAI algorithm can enhance the performance of readers for the detection of lung cancers on chest radiographs when used as second reader. KEY POINTS: • Reader study in the NLST dataset shows that AI algorithm had sensitivity benefit for residents and specificity benefit for radiologists for the detection of visible lung cancer. • With AI, radiology residents were able to recommend more chest CT examinations (54.7% vs 70.2%, p < 0.001) for patients with visible lung cancer. • With AI, radiologists recommended significantly less proportion of unnecessary chest CT examinations (16.4% vs. 11.7%, p < 0.001) in cancer-negative patients.

Indexed as

Artificial IntelligenceLung NeoplasmsAlgorithmsHumansLungRadiographic Image Interpretation, Computer-AssistedRadiographyRadiography, ThoracicSensitivity and SpecificityArtificial intelligenceComputer-assisted radiographic image interpretationLung cancerMass screeningRadiography

Identifiers

PMID34089072
OpenAlexW3165191451

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.