ArticleIEEE transactions on medical imaging2021
Interpretable Multimodal Fusion Networks Reveal Mechanisms of Brain Cognition.
Article in IEEE transactions on medical imaging, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 23 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
23 citing papers in PubMed, 57 citations in OpenAlex.
- Cross species single-nucleus characterization of mammillary body.Cell insight · 2026Article
- Enterocutaneous Fistula-Associated Sepsis and Mortality: Development and Validation of a Multimodal Artificial Intelligence Prediction Model.JMIR medical informatics · 2026Article
- Artificial intelligence as a surrogate brain: bridging neural dynamical models and data.National science review · 2026Review
- Integrated brain connectivity analysis with fMRI, DTI, and sMRI powered by interpretable graph neural networks.Medical image analysis · 2025Article
- Article
- A multimodal neural network with gradient blending improves predictions of survival and metastasis in sarcoma.NPJ precision oncology · 2024Article
- Explainable Multimodal Graph Isomorphism Network for Interpreting Sex Differences in Adolescent Neurodevelopment.Applied sciences (Basel, Switzerland) · 2024Article
- Interpretable Cognitive Ability Prediction: A Comprehensive Gated Graph Transformer Framework for Analyzing Functional Brain Networks.IEEE transactions on medical imaging · 2024Article
- Delineating Transdiagnostic Subtypes in Neurodevelopmental Disorders via Contrastive Graph Machine Learning of Brain Connectivity Patterns.bioRxiv : the preprint server for biology · 2024Article
- Applications of interpretable deep learning in neuroimaging: A comprehensive review.Imaging neuroscience (Cambridge, Mass.) · 2024Article
- ImageNomer: Description of a functional connectivity and omics analysis tool and case study identifying a race confound.Neuroimage. Reports · 2023Article
- Applications for Deep Learning in Epilepsy Genetic Research.International journal of molecular sciences · 2023Review
- Latent Similarity Identifies Important Functional Connections for Phenotype Prediction.IEEE transactions on bio-medical engineering · 2023Article
- A joint multi-modal learning method for early-stage knee osteoarthritis disease classification.Heliyon · 2023Article
- An explainable autoencoder with multi-paradigm fMRI fusion for identifying differences in dynamic functional connectivity during brain development.Neural networks : the official journal of the International Neural Network Society · 2023Article
- Data-driven multimodal fusion: approaches and applications in psychiatric research.Psychoradiology · 2023Review
- Distance Correlation-Based Brain Functional Connectivity Estimation and Non-Convex Multi-Task Learning for Developmental fMRI Studies.IEEE transactions on bio-medical engineering · 2022Article
- Multi-Modal Imaging Genetics Data Fusion via a Hypergraph-Based Manifold Regularization: Application to Schizophrenia Study.IEEE transactions on medical imaging · 2022Article
- Brain Functional Connectivity Analysis via Graphical Deep Learning.IEEE transactions on bio-medical engineering · 2022Article
- Decomposition-Based Correlation Learning for Multi-Modal MRI-Based Classification of Neuropsychiatric Disorders.Frontiers in neuroscience · 2022Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
11 authors at 5 institutions in 2 countries.
Funding
Abstract
The combination of multimodal imaging and genomics provides a more comprehensive way for the study of mental illnesses and brain functions. Deep network-based data fusion models have been developed to capture their complex associations, resulting in improved diagnosis of diseases. However, deep learning models are often difficult to interpret, bringing about challenges for uncovering biological mechanisms using these models. In this work, we develop an interpretable multimodal fusion model to perform automated diagnosis and result interpretation simultaneously. We name it Grad-CAM guided convolutional collaborative learning (gCAM-CCL), which is achieved by combining intermediate feature maps with gradient-based weights. The gCAM-CCL model can generate interpretable activation maps to quantify pixel-level contributions of the input features. Moreover, the estimated activation maps are class-specific, which can therefore facilitate the identification of biomarkers underlying different groups. We validate the gCAM-CCL model on a brain imaging-genetic study, and demonstrate its applications to both the classification of cognitive function groups and the discovery of underlying biological mechanisms. Specifically, our analysis results suggest that during task-fMRI scans, several object recognition related regions of interests (ROIs) are activated followed by several downstream encoding ROIs. In addition, the high cognitive group may have stronger neurotransmission signaling while the low cognitive group may have problems in brain/neuron development due to genetic variations.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.