ArticleRadiology. Artificial intelligence2022
Performance of a Chest Radiograph AI Diagnostic Tool for COVID-19: A Prospective Observational Study.
Abstract read
In one paragraph
Article in Radiology. Artificial intelligence, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 20 papers.
0numbers the graph read from it
0cells of the map it votes in
20citing papers in PubMed
–field-weighted citation impact
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
20 citing papers in PubMed.
- Review
- Comparison of chest X-ray radiography AI model to comorbidities for predicting intensive care unit admission for COVID-19.Journal of medical imaging (Bellingham, Wash.) · 2026Article
- Utility of the virtual imaging trials methodology for objective characterization of AI systems and training data.Journal of medical imaging (Bellingham, Wash.) · 2026Article
- Deep Learning Network Selection and Optimized Information Fusion for Enhanced COVID-19 Detection: A Literature Review.Diagnostics (Basel, Switzerland) · 2025Review
- Confidence-Aware Severity Assessment of Lung Disease from Chest X-Rays Using Deep Neural Network on a Multi-Reader Dataset.Journal of imaging informatics in medicine · 2025Article
- Impact of Radiologist Experience on AI Annotation Quality in Chest Radiographs: A Comparative Analysis.Diagnostics (Basel, Switzerland) · 2025Article
- Article
- Effects of artificial intelligence implementation on efficiency in medical imaging-a systematic literature review and meta-analysis.NPJ digital medicine · 2024Review
- Evolving and Novel Applications of Artificial Intelligence in Thoracic Imaging.Diagnostics (Basel, Switzerland) · 2024Review
- AI-assisted Analysis to Facilitate Detection of Humeral Lesions on Chest Radiographs.Radiology. Artificial intelligence · 2024Article
- Doctor's Orders-Why Radiologists Should Consider Adjusting Commercial Machine Learning Applications in Chest Radiography to Fit Their Specific Needs.Healthcare (Basel, Switzerland) · 2024Article
- Clinical decision support systems (CDSS) in assistance to COVID-19 diagnosis: A scoping review on types and evaluation methods.Health science reports · 2024Article
- Development and External Validation of an Artificial Intelligence-Based Method for Scalable Chest Radiograph Diagnosis: A Multi-Country Cross-Sectional Study.Research (Washington, D.C.) · 2024Article
- Multi-centre benchmarking of deep learning models for COVID-19 detection in chest x-rays.Frontiers in radiology · 2024Article
- Texture analysis of chest X-ray images for the diagnosis of COVID-19 pneumonia.Polish journal of radiology · 2024Article
- Artificial Intelligence Applied to Chest X-ray: A Reliable Tool to Assess the Differential Diagnosis of Lung Pneumonia in the Emergency Department.Diseases (Basel, Switzerland) · 2023Article
- Artificial Intelligence-enabled Decision Support in Surgery: State-of-the-art and Future Directions.Annals of surgery · 2023Article
- Estimating the Volume of Nodules and Masses on Serial Chest Radiography Using a Deep-Learning-Based Automatic Detection Algorithm: A Preliminary Study.Diagnostics (Basel, Switzerland) · 2023Article
- Classification of Monkeypox Images Using LIME-Enabled Investigation of Deep Convolutional Neural Network.Diagnostics (Basel, Switzerland) · 2023Article
- Unveiling COVID-19 Secrets: Harnessing Cytokines as Powerful Biomarkers for Diagnosis and Predicting Severity.Journal of inflammation research · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
17 authors.
Ju SunDepartment of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).ORCID https://orcid.org/0000-0002-2017-5903
Le PengDepartment of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).ORCID https://orcid.org/0000-0001-5728-5965
Taihui LiDepartment of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).ORCID https://orcid.org/0000-0002-3758-8923
Dyah AdilaDepartment of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).
Zach ZaimanDepartment of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).
Genevieve B Melton-MeauxDepartment of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).
Nicholas E IngrahamDepartment of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).ORCID https://orcid.org/0000-0002-0292-0594
Eric MurrayDepartment of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).
Daniel BoleyDepartment of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).ORCID https://orcid.org/0000-0002-2685-8449
Sean SwitzerDepartment of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).ORCID https://orcid.org/0000-0002-7018-5261
John L BurnsDepartment of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).ORCID https://orcid.org/0000-0003-3188-9298
Kun HuangDepartment of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).
Tadashi AllenDepartment of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).
Scott D SteenburgDepartment of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).ORCID https://orcid.org/0000-0002-8544-0616
Judy Wawira GichoyaDepartment of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).ORCID https://orcid.org/0000-0002-1097-316X
Erich Kummerfeld *Department of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).ORCID https://orcid.org/0000-0001-5342-7743
Christopher J Tignanelli *Department of Computer Science and Engineering (J.S., L.P., T.L., D.A., D.B.), Institute for Health Informatics (G.B.M.M., E.K., C.J.T.), Department of Surgery (G.B.M.M., C.J.T.), Department of Medicine, Division of Pulmonary and Critical Care (N.E.I.), Department of Medicine (S.S.), and Department of Radiology (T.A.), University of Minnesota, 420 Delaware St SE, Minneapolis, MN 55455; Departments of Computer Science (Z.Z.) and Radiology (J.W.G.), Emory University, Atlanta, Ga; M Health Fairview Informatics, Minneapolis, Minn (E.M.); The School of Medicine (J.L.B., K.H.) and Department of Radiology (S.D.S.), Indiana University, Indianapolis, Ind; and Department of Surgery, North Memorial Health Hospital, Robbinsdale, Minn (C.J.T.).ORCID https://orcid.org/0000-0003-3134-0924
Funding
University of Minnesota Clinical and Translational Science Institute (UMN CTSI)UL1TR002494 · NCATS · UNIVERSITY OF MINNESOTA · PI BLAZAR, BRUCE R, WEISDORF, DANIEL J · 2018 to 2022
$34.9MTRAINING-PULMONARY CELL &MOLECULAR BIOLOGY &PHYSIOLOGYT32HL007741 · NHLBI · UNIVERSITY OF MINNESOTA TWIN CITIES · PI DUDLEY, R. ADAMS, INGBAR, DAVID H · 1994 to 2023
$13.9MInstitutional Career Development CoreKL2TR002492 · NCATS · UNIVERSITY OF MINNESOTA · PI INGBAR, DAVID H · 2018 to 2022
$6.3MRADX TECH - CERES NANOSCIENCE INC NANOTRAP PARTICLE MANUFACTURING75N92020C00021 · NHLBI · CERES NANOSCIENCES, LLLP · PI DUNLAP, ROSS · 2021 to 2021
$4.0MAHRQ HHS K12 HS026379NCATS NIH HHS KL2 TR002492NCATS NIH HHS UL1 TR002494NHLBI NIH HHS 75N92020C00008NHLBI NIH HHS 75N92020C00021NHLBI NIH HHS T32 HL007741
Abstract
Purpose: To conduct a prospective observational study across 12 U.S. hospitals to evaluate real-time performance of an interpretable artificial intelligence (AI) model to detect COVID-19 on chest radiographs. Materials and Methods: A total of 95 363 chest radiographs were included in model training, external validation, and real-time validation. The model was deployed as a clinical decision support system, and performance was prospectively evaluated. There were 5335 total real-time predictions and a COVID-19 prevalence of 4.8% (258 of 5335). Model performance was assessed with use of receiver operating characteristic analysis, precision-recall curves, and F1 score. Logistic regression was used to evaluate the association of race and sex with AI model diagnostic accuracy. To compare model accuracy with the performance of board-certified radiologists, a third dataset of 1638 images was read independently by two radiologists. Results: Participants positive for COVID-19 had higher COVID-19 diagnostic scores than participants negative for COVID-19 (median, 0.1 [IQR, 0.0-0.8] vs 0.0 [IQR, 0.0-0.1], respectively; Conclusion: AI-based tools have not yet reached full diagnostic potential for COVID-19 and underperform compared with radiologist prediction.
Indexed as
Application DomainClassificationDiagnosisInfectionLung
Identifiers
What OpenQuestion holds
Textmetadata
Read underepoch 390
Registered trials
None linked
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.