Evidence map›Paper›PMID 34973101›Full record

ArticleEuropean radiology2022

Diagnostic effect of artificial intelligence solution for referable thoracic abnormalities on chest radiography: a multicenter respiratory outpatient diagnostic cohort study.

Kwang Nam Jin, Eun Young Kim, Young Jae Kim, Gi Pyo Lee, Hyungjin Kim, Sohee Oh, Yong Suk Kim, Ju Hyuck Han, Young Jun Cho

Registry-linked trialOpen access · hybridAbstract readMulticenter Study
In one paragraph

Article in European radiology, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT05489471 (A Prospective Study to Assess the Impact of an Artificial Intelligence System on Reporting of Chest X-rays, Evaluate the Ability of AI Driven Worklists to Improve Reporting Times and Improve Same Day CT Pathway for Suspected Lung Cancer), which is not on this map. Cited by 24 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
24citing papers in PubMed, 1 pooled it
4.5field-weighted citation impact, top 5% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

NCT05489471 unknown statusnot on this mapstarted 2023, after this paper: background citation

A Prospective Study to Assess the Impact of an Artificial Intelligence System on Reporting of Chest X-rays, Evaluate the Ability of AI Driven Worklists to Improve Reporting Times and Improve Same Day CT Pathway for Suspected Lung Cancer

TypeobservationalSponsorHull University Teaching Hospitals NHS TrustRan2023 to 2023Enrolled20,000ConditionsArtificial IntelligenceArmsArtificial intelligence review
3 · Its place in the literature

Who cites it

24 citing papers in PubMed, 1 synthesis or guideline pooled it, 33 citations in OpenAlex.

  1. Deep learning for pneumothorax diagnosis: a systematic review and meta-analysis.European respiratory review : an official journal of the European Respiratory Society · 2023
    Pooled it
  2. Trial
  3. Trial
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Article
  17. Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

9 authors at 5 institutions in 1 country.

Kwang Nam Jin *Department of Radiology, SMG-SNU Boramae Medical Center, Seoul, Korea.
Eun Young Kim *Department of Radiology, Gil Medical Center, Incheon, Korea.
Young Jae KimDepartment of Biomedical Engineering, Gachon University College of Medicine, Incheon, Korea.
Gi Pyo LeeDepartment of Biomedical Engineering, Gachon University College of Medicine, Incheon, Korea.
Hyungjin KimSeoul National University College of Medicine, Seoul, Korea.
Sohee OhDepartment of Biostatistics, SMG-SNU Boramae Medical Center, Seoul, Korea.
Yong Suk KimDepartment of Medical Artificial Intelligence, Konyang University, Daejeon, Korea.
Ju Hyuck HanDepartment of Medical Engineering, Konyang University, Daejeon, Korea.
Young Jun ChoDepartment of Radiology, Konyang University Hospital School of Medicine, Daejeon, Korea. cyj126@kyuh.ac.kr.ORCID http://orcid.org/0000-0002-0632-1011
Gachon University · KRKonyang University · KRSeoul National University · KRGachon University Gil Medical Center · KRSNUH SMG-SNU Boramae Medical Center · KR

Funding

Korea Health Industry Development Institute HI19C0847
6 · The paper itself

Abstract

objectivesWe aim ed to evaluate a commercial artificial intelligence (AI) solution on a multicenter cohort of chest radiographs and to compare physicians' ability to detect and localize referable thoracic abnormalities with and without AI assistance.

methodsIn this retrospective diagnostic cohort study, we investigated 6,006 consecutive patients who underwent both chest radiography and CT. We evaluated a commercially available AI solution intended to facilitate the detection of three chest abnormalities (nodule/masses, consolidation, and pneumothorax) against a reference standard to measure its diagnostic performance. Moreover, twelve physicians, including thoracic radiologists, board-certified radiologists, radiology residents, and pulmonologists, assessed a dataset of 230 randomly sampled chest radiographic images. The images were reviewed twice per physician, with and without AI, with a 4-week washout period. We measured the impact of AI assistance on observer's AUC, sensitivity, specificity, and the area under the alternative free-response ROC (AUAFROC).

resultsIn the entire set (n = 6,006), the AI solution showed average sensitivity, specificity, and AUC of 0.885, 0.723, and 0.867, respectively. In the test dataset (n = 230), the average AUC and AUAFROC across observers significantly increased with AI assistance (from 0.861 to 0.886; p = 0.003 and from 0.797 to 0.822; p = 0.003, respectively).

conclusionsThe diagnostic performance of the AI solution was found to be acceptable for the images from respiratory outpatient clinics. The diagnostic performance of physicians marginally improved with the use of AI solutions. Further evaluation of AI assistance for chest radiographs using a prospective design is required to prove the efficacy of AI assistance. KEY POINTS: • AI assistance for chest radiographs marginally improved physicians' performance in detecting and localizing referable thoracic abnormalities on chest radiographs. • The detection or localization of referable thoracic abnormalities by pulmonologists and radiology residents improved with the use of AI assistance.

Indexed as

Artificial IntelligenceRadiography, ThoracicCohort StudiesHumansOutpatientsProspective StudiesRadiographyRetrospective StudiesSensitivity and SpecificityArtificial intelligenceCohort studiesDiagnosisRadiographyThorax

Identifiers

PMID34973101
PMCPMC9038825
OpenAlexW4205355705

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.