ArticleJournal of biomedical informatics2023
A scoping review on multimodal deep learning in biomedical images and texts.
Article in Journal of biomedical informatics, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 15 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
15 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Application of artificial intelligence in esophageal surgery: a systematic review.Journal of robotic surgery · 2025Pooled it
- Application of artificial intelligence in the diagnosis of malignant digestive tract tumors: focusing on opportunities and challenges in endoscopy and pathology.Journal of translational medicine · 2025Pooled it
- From patient notes to prognostication: The revolutionary potential of event-based foundation models in orthopaedics.Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA · 2026Article
- Hybrid diagnostic framework for bone cancer detection using deep learning and radiomics analysis.Scientific reports · 2026Article
- Artificial Intelligence in Orthopaedics: Clinical Performance, Limitations, and Translational Readiness-A Review.Journal of clinical medicine · 2026Review
- Bridging modalities with AI: a review of AI advances in multimodal biomedical imaging.Communications engineering · 2026Review
- Multimodal deep learning outperforms clinical and brain region models in predicting stroke-associated pneumonia: an explainable AI study.Frontiers in neurology · 2026Article
- Multimodal Artificial Intelligence for Precision Critical Care: A Scoping Review.Health data science · 2026Review
- A multimodal synergistic model for personalized neoadjuvant immunochemotherapy in esophageal cancer.Cell reports. Medicine · 2025Article
- Standardized Corneal Topography-Driven AI for Orthokeratology Fitting: A Hybrid Deep/Machine Learning Approach With Enhanced Generalizability.Translational vision science & technology · 2025Article
- Combining Real and Synthetic Data to Overcome Limited Training Datasets in Multimodal Learning.medRxiv : the preprint server for health sciences · 2025Article
- Advancing Spine Fracture Detection: The Role of Artificial Intelligence in Clinical Practice.Korean journal of neurotrauma · 2025Review
- Deep learning-driven survival prediction in pan-cancer studies by integrating multimodal histology-genomic data.Briefings in bioinformatics · 2025Article
- Histopathology in focus: a review on explainable multi-modal approaches for breast cancer diagnosis.Frontiers in medicine · 2024Review
- Advancing healthcare through multimodal data fusion: a comprehensive review of techniques and applications.PeerJ. Computer science · 2024Article
Corrections and comments
- Update of
Authors and funding
7 authors.
Funding
Abstract
objectiveComputer-assisted diagnostic and prognostic systems of the future should be capable of simultaneously processing multimodal data. Multimodal deep learning (MDL), which involves the integration of multiple sources of data, such as images and text, has the potential to revolutionize the analysis and interpretation of biomedical data. However, it only caught researchers' attention recently. To this end, there is a critical need to conduct a systematic review on this topic, identify the limitations of current work, and explore future directions.
methodsIn this scoping review, we aim to provide a comprehensive overview of the current state of the field and identify key concepts, types of studies, and research gaps with a focus on biomedical images and texts joint learning, mainly because these two were the most commonly available data types in MDL research.
resultThis study reviewed the current uses of multimodal deep learning on five tasks: (1) Report generation, (2) Visual question answering, (3) Cross-modal retrieval, (4) Computer-aided diagnosis, and (5) Semantic segmentation.
conclusionOur results highlight the diverse applications and potential of MDL and suggest directions for future research in the field. We hope our review will facilitate the collaboration of natural language processing (NLP) and medical imaging communities and support the next generation of decision-making and computer-assisted diagnostic system development.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.