Evidence map›Paper›PMID 38641744›Full record

ArticleNature medicine2024

Demographic bias in misdiagnosis by computational pathology models.

Anurag Vaidya, Richard J Chen, Drew F K Williamson, Andrew H Song, Guillaume Jaume, Yuzhe Yang, Thomas Hartvigsen, Emma C Dyer, Ming Y Lu, Jana Lipkova and 3 more

Abstract read
PubMed Publisher
In one paragraph

Article in Nature medicine, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 68 papers.

0numbers the graph read from it
0cells of the map it votes in
68citing papers in PubMed
28.8field-weighted citation impact, top 1% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

68 citing papers in PubMed, 83 citations in OpenAlex.

  1. Article
  2. Article
  3. Review
  4. Review
  5. Review
  6. Article
  7. Article
  8. Article
  9. Article
  10. Review
  11. Article
  12. Article
  13. Review
  14. Review
  15. Article
  16. Article
  17. Article
  18. Article
  19. Review
  20. Article

8 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

13 authors at 4 institutions in 1 country.

Anurag Vaidya *Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
Richard J Chen *Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.ORCID 0000-0003-0389-1331
Drew F K Williamson *Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.ORCID 0000-0003-1745-8846
Andrew H SongDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
Guillaume JaumeDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
Yuzhe YangElectrical Engineering and Computer Science, MIT, Cambridge, MA, USA.ORCID 0000-0002-7634-8295
Thomas HartvigsenSchool of Data Science, University of Virginia, Charlottesville, VA, USA.
Emma C DyerT.H. Chan School of Public Health, Harvard University, Cambridge, MA, USA.
Ming Y LuDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.ORCID 0000-0003-0009-9699
Jana LipkovaDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
Muhammad ShabanDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
Tiffany Y ChenDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.ORCID 0000-0003-2546-2941
Faisal MahmoodDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA. faisalmahmood@bwh.harvard.edu.ORCID 0000-0001-7587-1562
Broad Institute · USBrigham and Women's Hospital · USHarvard University · USUniversity of Virginia · US

Funding

Training Program in Bioinformatics and Integrative GenomicsT32HG002295 · NHGRI · MASSACHUSETTS INSTITUTE OF TECHNOLOGY · PI Peter J Park · 2001 to 2026
$15.8M
Multimodal and Generative AI for PathologyR35GM138216 · NIGMS · BROAD INSTITUTE, INC. · PI Faisal Mahmood · 2020 to 2026
$3.4M
Oncopathology Training ProgramT32CA251062 · NCI · DANA-FARBER CANCER INST · PI JON C. ASTER, KATHLEEN H BURNS · 2020 to 2026
$2.4M
NCI NIH HHS T32 CA251062NIGMS NIH HHS R35 GM138216
6 · The paper itself

Abstract

Despite increasing numbers of regulatory approvals, deep learning-based computational pathology systems often overlook the impact of demographic factors on performance, potentially leading to biases. This concern is all the more important as computational pathology has leveraged large public datasets that underrepresent certain demographic groups. Using publicly available data from The Cancer Genome Atlas and the EBRAINS brain tumor atlas, as well as internal patient data, we show that whole-slide image classification models display marked performance disparities across different demographic groups when used to subtype breast and lung carcinomas and to predict IDH1 mutations in gliomas. For example, when using common modeling approaches, we observed performance gaps (in area under the receiver operating characteristic curve) between white and Black patients of 3.0% for breast cancer subtyping, 10.9% for lung cancer subtyping and 16.0% for IDH1 mutation prediction in gliomas. We found that richer feature representations obtained from self-supervised vision foundation models reduce performance variations between groups. These representations provide improvements upon weaker models even when those weaker models are combined with state-of-the-art bias mitigation strategies and modeling choices. Nevertheless, self-supervised vision foundation models do not fully eliminate these discrepancies, highlighting the continuing need for bias mitigation efforts in computational pathology. Finally, we demonstrate that our results extend to other demographic factors beyond patient race. Given these findings, we encourage regulatory and policy agencies to integrate demographic-stratified evaluation into their assessment guidelines.

Indexed as

GliomaLung NeoplasmsBiasBlack or African AmericanBlack PeopleDemographyDiagnostic ErrorsHumansIsocitrate DehydrogenaseWhiteIDH1 protein, humanIsocitrate Dehydrogenase

Identifiers

PMID38641744
OpenAlexW4394976513

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.