SynthesisMedical image analysis2021
Public Covid-19 X-ray datasets and their impact on model bias - A systematic review of a significant problem.
Synthesis in Medical image analysis, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 35 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
35 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Automated COVID-19 diagnosis and prognosis with medical imaging and who is publishing: a systematic review.Physical and engineering sciences in medicine · 2022Pooled it
- [Artificial intelligence in molecular pathology : Introduction and overview].Pathologie (Heidelberg, Germany) · 2026Review
- Artificial intelligence for chest radiography: an overview of techniques, challenges, and future directions.npj health systems · 2026Review
- Evaluation framework for assessing the quality and safety of artificial intelligence solutions for the acute care surgeon: a narrative review from the Eastern Association for the Surgery of Trauma quality, safety, and outcomes committee.Trauma surgery & acute care open · 2026Review
- CIRCA: comprehensible online system in support of chest X-rays-based screening by COVID-19 example.Scientific reports · 2025Article
- Tailoring task arithmetic to address bias in models trained on multi-institutional datasets.Journal of biomedical informatics · 2025Article
- Deep Learning Network Selection and Optimized Information Fusion for Enhanced COVID-19 Detection: A Literature Review.Diagnostics (Basel, Switzerland) · 2025Review
- A Pipeline for Automated Quality Control of Chest Radiographs.Radiology. Artificial intelligence · 2025Article
- Assessing the documentation of publicly available medical image and signal datasets and their impact on bias using the BEAMRAD tool.Scientific reports · 2024Article
- Distribution-based detection of radiographic changes in pneumonia patterns: A COVID-19 case study.Heliyon · 2024Article
- Deep Learning for Pneumonia Detection in Chest X-ray Images: A Comprehensive Survey.Journal of imaging · 2024Review
- Generalizable disease detection using model ensemble on chest X-ray images.Scientific reports · 2024Article
- RETRACTED: An inherently interpretable deep learning model for local explanations using visual concepts.PloS one · 2024Article
- Pulmonary abnormality screening on chest x-rays from different machine specifications: a generalized AI-based image manipulation pipeline.European radiology experimental · 2023Article
- MultiCOVID: a multi modal deep learning approach for COVID-19 diagnosis.Scientific reports · 2023Article
- Computer-aided diagnosis of chest X-ray for COVID-19 diagnosis in external validation study by radiologists with and without deep learning system.Scientific reports · 2023Article
- Pathological changes or technical artefacts? The problem of the heterogenous databases in COVID-19 CXR image analysis.Computer methods and programs in biomedicine · 2023Article
- POLCOVID: a multicenter multiclass chest X-ray database (Poland, 2020-2021).Scientific data · 2023Article
- Bias Analysis in Healthcare Time Series (BAHT) Decision Support Systems from Meta Data.Journal of healthcare informatics research · 2023Article
- The COVID-19 Pandemic: How Technology Is Reshaping Public Health and Medicine.Bioengineering (Basel, Switzerland) · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Computer-aided-diagnosis and stratification of COVID-19 based on chest X-ray suffers from weak bias assessment and limited quality-control. Undetected bias induced by inappropriate use of datasets, and improper consideration of confounders prevents the translation of prediction models into clinical practice. By adopting established tools for model evaluation to the task of evaluating datasets, this study provides a systematic appraisal of publicly available COVID-19 chest X-ray datasets, determining their potential use and evaluating potential sources of bias. Only 9 out of more than a hundred identified datasets met at least the criteria for proper assessment of risk of bias and could be analysed in detail. Remarkably most of the datasets utilised in 201 papers published in peer-reviewed journals, are not among these 9 datasets, thus leading to models with high risk of bias. This raises concerns about the suitability of such models for clinical use. This systematic review highlights the limited description of datasets employed for modelling and aids researchers to select the most suitable datasets for their task.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.