ArticleBMJ open2021
Do comprehensive deep learning algorithms suffer from hidden stratification? A retrospective study on pneumothorax detection in chest radiography.
Article in BMJ open, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 9 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
9 citing papers in PubMed, 1 synthesis or guideline pooled it, 15 citations in OpenAlex.
- Deep learning for pneumothorax diagnosis: a systematic review and meta-analysis.European respiratory review : an official journal of the European Respiratory Society · 2023Pooled it
- Predictive performance precision analysis in medicine: identification of low-confidence predictions at patient and profile levels (MED3pa I).Journal of the American Medical Informatics Association : JAMIA · 2026Article
- Gaps in artificial intelligence research for rural health in the United States: a scoping review.Journal of the American Medical Informatics Association : JAMIA · 2026Article
- Diagnostic Codes in AI Prediction Models and Label Leakage of Same-Admission Clinical Outcomes.JAMA network open · 2025Article
- Development of an AI model for pneumothorax imaging: Dataset and model optimization strategies for real-world deployment.European journal of radiology open · 2025Article
- Deep learning for tubes and lines detection in critical illness: Generalizability and comparison with residents.European journal of radiology open · 2024Article
- Better performance of deep learning pulmonary nodule detection using chest radiography with pixel level labels in reference to computed tomography: data quality matters.Scientific reports · 2024Article
- Radiomics-based decision support tool assists radiologists in small lung nodule classification and improves lung cancer early diagnosis.British journal of cancer · 2023Article
- Analysis of Line and Tube Detection Performance of a Chest X-ray Deep Learning Model to Evaluate Hidden Stratification.Diagnostics (Basel, Switzerland) · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
11 authors at 5 institutions in 2 countries.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectivesTo evaluate the ability of a commercially available comprehensive chest radiography deep convolutional neural network (DCNN) to detect simple and tension pneumothorax, as stratified by the following subgroups: the presence of an intercostal drain; rib, clavicular, scapular or humeral fractures or rib resections; subcutaneous emphysema and erect versus non-erect positioning. The hypothesis was that performance would not differ significantly in each of these subgroups when compared with the overall test dataset.
designA retrospective case-control study was undertaken.
settingCommunity radiology clinics and hospitals in Australia and the USA.
participantsA test dataset of 2557 chest radiography studies was ground-truthed by three subspecialty thoracic radiologists for the presence of simple or tension pneumothorax as well as each subgroup other than positioning. Radiograph positioning was derived from radiographer annotations on the images. OUTCOME MEASURES: DCNN performance for detecting simple and tension pneumothorax was evaluated over the entire test set, as well as within each subgroup, using the area under the receiver operating characteristic curve (AUC). A difference in AUC of more than 0.05 was considered clinically significant.
resultsWhen compared with the overall test set, performance of the DCNN for detecting simple and tension pneumothorax was statistically non-inferior in all subgroups. The DCNN had an AUC of 0.981 (0.976-0.986) for detecting simple pneumothorax and 0.997 (0.995-0.999) for detecting tension pneumothorax.
conclusionsHidden stratification has significant implications for potential failures of deep learning when applied in clinical practice. This study demonstrated that a comprehensively trained DCNN can be resilient to hidden stratification in several clinically meaningful subgroups in detecting pneumothorax.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.