Evidence map›Paper›PMID 36220072›Full record

ReviewCancer cell2022

Artificial intelligence for multimodal data integration in oncology.

Jana Lipkova, Richard J Chen, Bowen Chen, Ming Y Lu, Matteo Barbieri, Daniel Shao, Anurag J Vaidya, Chengkuan Chen, Luoting Zhuang, Drew F K Williamson and 3 more

Registry-linked trialAbstract readReview
PubMed Full text at PMC Publisher
In one paragraph

Review in Cancer cell, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT07543263 (An Artificial Intelligence System for Multimodal, Multi-class Diagnosis of Pancreatic Cystic Lesions Based on Endoscopic Ultrasonography), which is not on this map. Cited by 353 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
353citing papers in PubMed, 2 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

NCT07543263 recruitingnot on this mapstarted 2026, after this paper: background citation

An Artificial Intelligence System for Multimodal, Multi-class Diagnosis of Pancreatic Cystic Lesions Based on Endoscopic Ultrasonography

TypeobservationalSponsorQilu Hospital of Shandong UniversityRan2026 to 2028Enrolled176ConditionsPancreatic Cystic Lesion (PCL)ArmsiEUS-PCL(intelligent endoscopic ultrasound system- pancreatic cystic lesion)
3 · Its place in the literature

Who cites it

353 citing papers in PubMed, 2 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Trial
  4. Article
  5. Article
  6. Review
  7. Spinal morphology-based multimodal AI for predicting pulmonary dysfunction in adolescent idiopathic scoliosis.European spine journal : official publication of the European Spine Society, the European Spinal Deformity Society, and the European Section of the Cervical Spine Research Society · 2026
    Article
  8. Article
  9. Review
  10. Article
  11. Article
  12. Review
  13. Article
  14. Review
  15. Article
  16. Review
  17. Article
  18. Observational
  19. Article
  20. Review

293 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

13 authors.

Jana LipkovaDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.
Richard J ChenDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.
Bowen ChenDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Department of Computer Science, Harvard University, Cambridge, MA, USA.
Ming Y LuDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology (MIT), Cambridge, MA, USA.
Matteo BarbieriDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
Daniel ShaoDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Harvard-MIT Health Sciences and Technology (HST), Cambridge, MA, USA.
Anurag J VaidyaDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Harvard-MIT Health Sciences and Technology (HST), Cambridge, MA, USA.
Chengkuan ChenDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.
Luoting ZhuangDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA.
Drew F K WilliamsonDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.
Muhammad ShabanDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.
Tiffany Y ChenDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.
Faisal MahmoodDepartment of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Harvard Data Science Initiative, Harvard University, Cambridge, MA, USA. Electronic address: faisalmahmood@bwh.harvard.edu.

Funding

Training Program in Bioinformatics and Integrative GenomicsT32HG002295 · NHGRI · MASSACHUSETTS INSTITUTE OF TECHNOLOGY · PI Peter J Park · 2001 to 2026
$15.8M
Multimodal and Generative AI for PathologyR35GM138216 · NIGMS · BROAD INSTITUTE, INC. · PI Faisal Mahmood · 2020 to 2026
$3.4M
Oncopathology Training ProgramT32CA251062 · NCI · DANA-FARBER CANCER INST · PI JON C. ASTER, KATHLEEN H BURNS · 2020 to 2026
$2.4M
NCI NIH HHS T32 CA251062NHGRI NIH HHS T32 HG002295NIGMS NIH HHS R35 GM138216
6 · The paper itself

Abstract

In oncology, the patient state is characterized by a whole spectrum of modalities, ranging from radiology, histology, and genomics to electronic health records. Current artificial intelligence (AI) models operate mainly in the realm of a single modality, neglecting the broader clinical context, which inevitably diminishes their potential. Integration of different data modalities provides opportunities to increase robustness and accuracy of diagnostic and prognostic models, bringing AI closer to clinical practice. AI models are also capable of discovering novel patterns within and across modalities suitable for explaining differences in patient outcomes or treatment resistance. The insights gleaned from such models can guide exploration studies and contribute to the discovery of novel biomarkers and therapeutic targets. To support these advances, here we present a synopsis of AI methods and strategies for multimodal data fusion and association discovery. We outline approaches for AI interpretability and directions for AI-driven exploration through multimodal data interconnections. We examine challenges in clinical adoption and discuss emerging solutions.

Indexed as

Artificial IntelligenceRadiologyElectronic Health RecordsGenomicsHumansMedical OncologyAI in oncologydeep learningdeep learning in oncologymultimodal AImultimodal fusionmultimodal integration

Identifiers

PMID36220072
PMCPMC10655164

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.