ArticlePloS one2020
Analyzing inter-reader variability affecting deep ensemble learning for COVID-19 detection in chest radiographs.
Article in PloS one, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 31 papers, 3 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
31 citing papers in PubMed, 3 syntheses or guidelines pooled it.
- Advances in Deep Learning for Tuberculosis Screening using Chest X-rays: The Last 5 Years Review.Journal of medical systems · 2022Pooled it
- Automated COVID-19 diagnosis and prognosis with medical imaging and who is publishing: a systematic review.Physical and engineering sciences in medicine · 2022Pooled it
- X-Ray Equipped with Artificial Intelligence: Changing the COVID-19 Diagnostic Paradigm during the Pandemic.BioMed research international · 2021Pooled it
- Impact of Radiologist Experience on AI Annotation Quality in Chest Radiographs: A Comparative Analysis.Diagnostics (Basel, Switzerland) · 2025Article
- Computer-Aided Lung Auscultation Screening and Radiographic Evaluation of Pediatric Pneumonia.Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference · 2024Article
- COVID-19 Hierarchical Classification Using a Deep Learning Multi-Modal.Sensors (Basel, Switzerland) · 2024Article
- A brief review and scientometric analysis on ensemble learning methods for handling COVID-19.Heliyon · 2024Article
- Improving Computer-Aided Thoracic Disease Diagnosis through Comparative Analysis Using Chest X-ray Images Taken at Different Times.Sensors (Basel, Switzerland) · 2024Article
- A deep-learning-based framework for identifying and localizing multiple abnormalities and assessing cardiomegaly in chest X-ray.Nature communications · 2024Article
- Noise-induced modality-specific pretext learning for pediatric chest X-ray image classification.Frontiers in artificial intelligence · 2024Article
- COVID-19 screening in low resource settings using artificial intelligence for chest radiographs and point-of-care blood tests.Scientific reports · 2023Article
- Crowd-Sourced Deep Learning for Intracranial Hemorrhage Identification: Wisdom of Crowds or Laissez-Faire.AJNR. American journal of neuroradiology · 2023Article
- A 178-clinical-center experiment of integrating AI solutions for lung pathology diagnosis.Scientific reports · 2023Article
- Ensemble Learning of Multiple Models Using Deep Learning for Multiclass Classification of Ultrasound Images of Hepatic Masses.Bioengineering (Basel, Switzerland) · 2023Article
- Review on the Evaluation and Development of Artificial Intelligence for COVID-19 Containment.Sensors (Basel, Switzerland) · 2023Review
- Assessing Inter-Annotator Agreement for Medical Image Segmentation.IEEE access : practical innovations, open solutions · 2023Article
- Data Characterization for Reliable AI in Medicine.Recent trends in image processing and pattern recognition : 5th International Conference, RTIP2R 2022, Kingsville, TX, USA, December 01-02, 2022, revised selected papers. International Conference on Recent Trends in Image Processing and... · 2023Article
- Editorial on Special Issue "Artificial Intelligence in Image-Based Screening, Diagnostics, and Clinical Care of Cardiopulmonary Diseases".Diagnostics (Basel, Switzerland) · 2022Article
- Deep Ensemble Learning for the Automatic Detection of Pneumoconiosis in Coal Worker's Chest X-ray Radiography.Journal of clinical medicine · 2022Article
- Think positive: An interpretable neural network for image recognition.Neural networks : the official journal of the International Neural Network Society · 2022Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Data-driven deep learning (DL) methods using convolutional neural networks (CNNs) demonstrate promising performance in natural image computer vision tasks. However, their use in medical computer vision tasks faces several limitations, viz., (i) adapting to visual characteristics that are unlike natural images; (ii) modeling random noise during training due to stochastic optimization and backpropagation-based learning strategy; (iii) challenges in explaining DL black-box behavior to support clinical decision-making; and (iv) inter-reader variability in the ground truth (GT) annotations affecting learning and evaluation. This study proposes a systematic approach to address these limitations through application to the pandemic-caused need for Coronavirus disease 2019 (COVID-19) detection using chest X-rays (CXRs). Specifically, our contribution highlights significant benefits obtained through (i) pretraining specific to CXRs in transferring and fine-tuning the learned knowledge toward improving COVID-19 detection performance; (ii) using ensembles of the fine-tuned models to further improve performance over individual constituent models; (iii) performing statistical analyses at various learning stages for validating results; (iv) interpreting learned individual and ensemble model behavior through class-selective relevance mapping (CRM)-based region of interest (ROI) localization; and, (v) analyzing inter-reader variability and ensemble localization performance using Simultaneous Truth and Performance Level Estimation (STAPLE) methods. We find that ensemble approaches markedly improved classification and localization performance, and that inter-reader variability and performance level assessment helps guide algorithm design and parameter optimization. To the best of our knowledge, this is the first study to construct ensembles, perform ensemble-based disease ROI localization, and analyze inter-reader variability and algorithm performance for COVID-19 detection in CXRs.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.