ArticlePLOS digital health2022
Reproducibility of deep learning in digital pathology whole slide image analysis.
Article in PLOS digital health, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 14 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
14 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Global trends, translational gaps, and emerging research directions inFrontiers in fungal biology · 2026Pooled it
- Benefits of high-magnification and color information in computational pathology-An exploratory review.Journal of pathology informatics · 2026Review
- Challenges and opportunities of implementing open-source digital pathology in sub-Saharan Africa: Lessons learned from deploying QuPath for breast cancer biomarker assessment in Cameroon.Journal of pathology informatics · 2026Article
- Comparative analysis of whole-slide scanner tissue detection algorithms: Implications for scan area, scan time, and file size in high-volume digital pathology workflows.Journal of pathology informatics · 2026Article
- Generative AI in Veterinary Pathology: Feasibility of a GPT-Based Assistive Tool for Gross, Cytologic, and Histopathologic Assessment of Canine Cutaneous Neoplasms-A Pilot Study.Animals : an open access journal from MDPI · 2026Article
- Review
- Artificial intelligence in ovarian cancer: advancing in precision diagnosis and clinical management.Frontiers in immunology · 2026Review
- Artificial intelligence and computational prediction models for risk stratification, treatment response, and outcomes in colorectal cancer: a narrative review.Frontiers in oncology · 2026Review
- Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities.Journal of pathology informatics · 2024Review
- Performance of externally validated machine learning models based on histopathology images for the diagnosis, classification, prognosis, or treatment outcome prediction in female breast cancer: A systematic review.Journal of pathology informatics · 2024Review
- Reproducibility and explainability in digital pathology: The need to make black-box artificial intelligence systems more transparent.Journal of public health research · 2024Article
- Artificial Intelligence Models for the Detection of Microsatellite Instability from Whole-Slide Imaging of Colorectal Cancer.Diagnostics (Basel, Switzerland) · 2024Review
- Magnifying Networks for Histopathological Images with Billions of Pixels.Diagnostics (Basel, Switzerland) · 2024Article
- The NCI Imaging Data Commons as a platform for reproducible research in computational pathology.Computer methods and programs in biomedicine · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
For a method to be widely adopted in medical research or clinical practice, it needs to be reproducible so that clinicians and regulators can have confidence in its use. Machine learning and deep learning have a particular set of challenges around reproducibility. Small differences in the settings or the data used for training a model can lead to large differences in the outcomes of experiments. In this work, three top-performing algorithms from the Camelyon grand challenges are reproduced using only information presented in the associated papers and the results are then compared to those reported. Seemingly minor details were found to be critical to performance and yet their importance is difficult to appreciate until the actual reproduction is attempted. We observed that authors generally describe the key technical aspects of their models well but fail to maintain the same reporting standards when it comes to data preprocessing which is essential to reproducibility. As an important contribution of the present study and its findings, we introduce a reproducibility checklist that tabulates information that needs to be reported in histopathology ML-based work in order to make it reproducible.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.