Evidence map›Paper›PMID 38504017›Full record

ArticleNature medicine2024

A visual-language foundation model for computational pathology.

Ming Y Lu, Bowen Chen, Drew F K Williamson, Richard J Chen, Ivy Liang, Tong Ding, Guillaume Jaume, Igor Odintsov, Long Phi Le, Georg Gerber and 3 more

Abstract read
In one paragraph

Article in Nature medicine, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 412 papers.

0numbers the graph read from it
0cells of the map it votes in
412citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

412 citing papers in PubMed.

  1. Article
  2. Review
  3. Article
  4. Article
  5. DeepLNM: A Multimodal Model for Predicting Lymph Node Metastasis in Colorectal Cancer.Advanced science (Weinheim, Baden-Wurttemberg, Germany) · 2026
    Article
  6. Review
  7. Article
  8. Article
  9. Review
  10. A Facial Foundation Model for Clinical Biomarker Prediction and Real-World Mobile Deployment.Advanced science (Weinheim, Baden-Wurttemberg, Germany) · 2026
    Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Review
  17. Article
  18. Article
  19. Article
  20. Article

352 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

13 authors.

Ming Y Lu *Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.ORCID 0000-0003-0009-9699
Bowen Chen *Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
Drew F K Williamson *Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.ORCID 0000-0003-1745-8846
Richard J ChenDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.ORCID 0000-0003-0389-1331
Ivy LiangDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
Tong DingDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
Guillaume JaumeDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
Igor OdintsovDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
Long Phi LeDepartment of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.
Georg GerberDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.ORCID 0000-0002-9149-5509
Anil V ParwaniDepartment of Pathology, Wexner Medical Center, Ohio State University, Columbus, OH, USA.
Andrew ZhangDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.ORCID 0000-0002-9432-2793
Faisal MahmoodDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA. faisalmahmood@bwh.harvard.edu.ORCID 0000-0001-7587-1562

Funding

STRUCTURE-FUNCTION RELATIONSHIPS IN THE ALIMENTARY TRACTP30DK034854 · NIDDK · HARVARD UNIVERSITY (MEDICAL SCHOOL) · PI JONATHAN C KAGAN · 1986 to 2026
$32.4M
Training Program in Bioinformatics and Integrative GenomicsT32HG002295 · NHGRI · MASSACHUSETTS INSTITUTE OF TECHNOLOGY · PI Peter J Park · 2001 to 2026
$15.8M
Multimodal and Generative AI for PathologyR35GM138216 · NIGMS · BROAD INSTITUTE, INC. · PI Faisal Mahmood · 2020 to 2026
$3.4M
Oncopathology Training ProgramT32CA251062 · NCI · DANA-FARBER CANCER INST · PI JON C. ASTER, KATHLEEN H BURNS · 2020 to 2026
$2.4M
Probabilistic deep learning models and integrated biological experiments for analyzing dynamic and heterogeneous microbiomesR35GM149270 · NIGMS · BRIGHAM AND WOMEN'S HOSPITAL · PI Georg Kurt Gerber · 2023 to 2026
$1.8M
NCI NIH HHS T32 CA251062NHGRI NIH HHS T32 HG002295NIDDK NIH HHS P30 DK034854NIGMS NIH HHS R35 GM138216NIGMS NIH HHS R35 GM149270
6 · The paper itself

Abstract

The accelerated adoption of digital pathology and advances in deep learning have enabled the development of robust models for various pathology tasks across a diverse array of diseases and patient cohorts. However, model training is often difficult due to label scarcity in the medical domain, and a model's usage is limited by the specific task and disease for which it is trained. Additionally, most models in histopathology leverage only image data, a stark contrast to how humans teach each other and reason about histopathologic entities. We introduce CONtrastive learning from Captions for Histopathology (CONCH), a visual-language foundation model developed using diverse sources of histopathology images, biomedical text and, notably, over 1.17 million image-caption pairs through task-agnostic pretraining. Evaluated on a suite of 14 diverse benchmarks, CONCH can be transferred to a wide range of downstream tasks involving histopathology images and/or text, achieving state-of-the-art performance on histology image classification, segmentation, captioning, and text-to-image and image-to-text retrieval. CONCH represents a substantial leap over concurrent visual-language pretrained systems for histopathology, with the potential to directly facilitate a wide array of machine learning-based workflows requiring minimal or no further supervised fine-tuning.

Indexed as

LanguageMachine LearningHumansWorkflow

Identifiers

PMID38504017
PMCPMC11384335

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.