A hybrid ST-ViT-driven multimodal architecture combining spatiotemporal MRI patterns and radiomic features for enhanced prediction of pCR in neoadjuvant breast cancer therapy.

Breast cancer response prediction plays a critical role in treatment planning, especially for identifying patients likely to achieve pathologic complete response (pCR). Traditional approaches rely primarily on baseline clinical variables or static imaging features, limiting their ability to capture the complex biological and temporal dynamics of tumor evolution during therapy. This study presents a spatiotemporal multimodal framework that integrates quantitative clinicopathological variables, radiomics, and longitudinal DCE-MRI to enhance the prediction of pCR. We employ a Spatiotemporal Vision Transformer (ST-ViT) to model tumor evolution across four imaging time points and fuse it with quantitative radiomic and clinical features. The proposed framework captures both spatial heterogeneity and treatment-induced temporal changes, offering a comprehensive representation of tumor biology. Texture-based radiomic analysis reveals meaningful differences between pCR and non-pCR tumors, while enhancement-curve dynamics further highlight early perfusion and washout patterns linked to treatment sensitivity. The integrated spatiotemporal multimodal model demonstrates strong discriminatory power, yielding an AUC of 0.98 during training and 0.96 on the held-out test set. These results highlight the model’s ability to leverage dynamic MRI signatures, radiomic texture descriptors, and clinical features distinguish responders from non-responders effectively. By capturing both spatial and temporal tumor evolution, the framework offers a robust and clinically meaningful tool for early identification of treatment-sensitive phenotypes and supports precision-driven neoadjuvant therapy planning.