End-to-end prediction of clinical outcomes in head and neck squamous cell carcinoma with foundation model-based multiple instance learning.

Background: Foundation models have shown promise in medical AI by learning flexible features from large datasets, offering new opportunities for improving endpoint prediction. However, usage of foundation models for endpoint prediction using routine imaging in head and neck squamous cell carcinoma patients remains unexplored. Within this study, we evaluated the potential of foundation-model based multiple instance learning for prediction of 2-year overall survival, locoregional control and freedom from distant metastasis across three external head and neck squamous cell carcinoma patient cohorts using 2D, multiview and 3D approaches while comparing prediction and stratification performance with handcrafted radiomics and clinical baselines. Results: 2D multiple-instance learning models achieved 2-year test area under the receiver-operator curve (AUROC) range of 0.75-0.84 for 2-year overall survival, 0.66-0.75 for 2-year locoregional control and 0.71-0.78 for 2-year freedom from distant metastasis across three different external cohorts, outperforming multiview and 3D multiple instance learning models (AUROC range: 0.50-0.77, p Conclusions: FM-based 2D MIL demonstrates promise in HNSCC risk prediction as well as stratification of clinical outcomes. The models match or outperform radiomics baselines, learning clinically-related patterns and showing enhancement of clinical baselines in non-human papilloma virus positive patients. Supplementary Information: The online version contains supplementary material available at 10.1186/s44398-025-00003-8.