Evidence map›Paper›PMID 36323656›Full record

ArticleNature communications2022

Uncertainty-informed deep learning models enable high-confidence predictions for digital histopathology.

James M Dolezal, Andrew Srisuwananukorn, Dmitry Karpeyev, Siddhi Ramesh, Sara Kochanny, Brittany Cody, Aaron S Mansfield, Sagar Rakshit, Radhika Bansal, Melanie C Bois and 6 more

Open access · goldAbstract read
In one paragraph

Article in Nature communications, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 52 papers.

0numbers the graph read from it
0cells of the map it votes in
52citing papers in PubMed
14.8field-weighted citation impact, top 1% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

52 citing papers in PubMed, 115 citations in OpenAlex.

  1. Article
  2. Article
  3. Review
  4. Article
  5. Review
  6. Article
  7. Review
  8. Article
  9. Article
  10. Article
  11. Review
  12. Article
  13. Article
  14. Redefining oropharyngeal cancer in the HPV era: integrating precision medicine and immunotherapeutic frontiers.Clinical & translational oncology : official publication of the Federation of Spanish Oncology Societies and of the National Cancer Institute of Mexico · 2026
    Review
  15. Review
  16. Article
  17. Article
  18. Article
  19. Review
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

16 authors at 6 institutions in 1 country.

James M DolezalSection of Hematology/Oncology, Department of Medicine, University of Chicago Medical Center, Chicago, IL, USA.ORCID 0000-0002-6788-7203
Andrew SrisuwananukornTisch Cancer Institute, Icahn School of Medicine at Mount Sinai, New York, NY, USA.ORCID 0000-0002-8736-8726
Dmitry KarpeyevDV Group, LLC, Chicago, IL, USA.
Siddhi RameshSection of Hematology/Oncology, Department of Medicine, University of Chicago Medical Center, Chicago, IL, USA.
Sara KochannySection of Hematology/Oncology, Department of Medicine, University of Chicago Medical Center, Chicago, IL, USA.
Brittany CodyDepartment of Pathology, University of Chicago, Chicago, IL, USA.
Aaron S MansfieldDivision of Medical Oncology, Mayo Clinic, Rochester, MN, USA.ORCID 0000-0002-9483-6903
Sagar RakshitDivision of Medical Oncology, Mayo Clinic, Rochester, MN, USA.
Radhika BansalDivision of Medical Oncology, Mayo Clinic, Rochester, MN, USA.ORCID 0000-0003-0964-098X
Melanie C BoisDepartment of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA.
Aaron O BungumDivisions of Pulmonary Medicine and Critical Care, Mayo Clinic, Rochester, MN, USA.
Jefree J SchulteDepartment of Pathology and Laboratory Medicine, University of Wisconsin at Madison, Madison, WN, USA.ORCID 0000-0003-3920-0301
Everett E VokesSection of Hematology/Oncology, Department of Medicine, University of Chicago Medical Center, Chicago, IL, USA.
Marina Chiara GarassinoSection of Hematology/Oncology, Department of Medicine, University of Chicago Medical Center, Chicago, IL, USA.
Aliya N HusainDepartment of Pathology, University of Chicago, Chicago, IL, USA.
Alexander T PearsonSection of Hematology/Oncology, Department of Medicine, University of Chicago Medical Center, Chicago, IL, USA. apearson5@medicine.bsd.uchicago.edu.ORCID 0000-0003-2801-7456
University of Chicago Medical Center · USMayo Clinic in Arizona · USUniversity of Chicago · USIcahn School of Medicine at Mount Sinai · USMayo Clinic · USUniversity of Wisconsin–Madison · US

Funding

Women's Cancer ProgramP30CA015083 · NCI · MAYO CLINIC ROCHESTER · PI Felicity Enders · 1985 to 2026
$151.3M
BASIC RESEARCH TRAINING IN MEDICAL ONCOLOGYT32CA009566 · NCI · UNIVERSITY OF CHICAGO · PI OLUFUNMILAYO F. OLOPADE · 1987 to 2026
$10.5M
Multiscale Computational Models Guided By Emerging Cellular Dynamics Quantification For Predicting Optimum Immune Checkpoint And Targeted Therapy SchedulesU01CA243075 · NCI · UNIVERSITY OF MICHIGAN AT ANN ARBOR · PI JACKSON, TRACHETTE · 2020 to 2024
$2.6M
Blood-based biomarker of therapeutic efficacy for mesotheliomaR21CA251923 · NCI · MAYO CLINIC ROCHESTER · PI MANSFIELD, AARON SCOTT · 2020 to 2020
$409k
Deep learning for oral premalignancy evaluationR56DE030958 · NIDCR · UNIVERSITY OF CHICAGO · PI PEARSON, ALEXANDER THOMAS · 2021 to 2021
$331k
NCI NIH HHS P30 CA015083NCI NIH HHS R21 CA251923NCI NIH HHS T32 CA009566NCI NIH HHS U01 CA243075NIDCR NIH HHS R56 DE030958
6 · The paper itself

Abstract

A model's ability to express its own predictive uncertainty is an essential attribute for maintaining clinical user confidence as computational biomarkers are deployed into real-world medical settings. In the domain of cancer digital histopathology, we describe a clinically-oriented approach to uncertainty quantification for whole-slide images, estimating uncertainty using dropout and calculating thresholds on training data to establish cutoffs for low- and high-confidence predictions. We train models to identify lung adenocarcinoma vs. squamous cell carcinoma and show that high-confidence predictions outperform predictions without uncertainty, in both cross-validation and testing on two large external datasets spanning multiple institutions. Our testing strategy closely approximates real-world application, with predictions generated on unsupervised, unannotated slides using predetermined thresholds. Furthermore, we show that uncertainty thresholding remains reliable in the setting of domain shift, with accurate high-confidence predictions of adenocarcinoma vs. squamous cell carcinoma for out-of-distribution, non-lung cancer cohorts.

Indexed as

AdenocarcinomaCarcinoma, Squamous CellDeep LearningHumansUncertainty

Identifiers

PMID36323656
PMCPMC9630455
OpenAlexW4308149258

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.