Multimodal deep learning: tumor and visceral fat impact on colorectal cancer occult peritoneal metastasis.

objectivesThis study proposes a multimodal deep learning (DL) approach to investigate the impact of tumors and visceral fat on occult peritoneal metastasis in colorectal cancer (CRC) patients.

methodsWe developed a DL model named Multi-scale Feature Fusion Network (MSFF-Net) based on ResNet18, which extracted features of tumors and visceral fat from the longest diameter tumor section and the third lumbar vertebra level (L3) in preoperative CT scans of CRC patients. Logistic regression analysis was applied to patients' clinical data that integrated with DL features. A random forest (RF) classifier was established to evaluate the MSFF-Net's performance on internal and external test sets and compare it with radiologists' performance.

resultsThe model incorporating fat features outperformed the single tumor modality in the internal test set. Combining clinical information with DL provided the best diagnostic performance for predicting peritoneal metastasis in CRC patients. The AUCs were 0.941 (95% CI: [0.891, 0.986], p = 0.03) for the internal test set and 0.911 (95% CI: [0.857, 0.971], p = 0.013) for the external test set. CRC patients with peritoneal metastasis had a higher visceral adipose tissue index (VATI) compared to those without. Maximum tumor diameter and VATI were identified as independent prognostic factors for peritoneal metastasis. Grad-CAM decision regions corresponded with the independent prognostic factors identified by logistic regression analysis.

conclusionThe study confirms the network features of tumors and visceral fat significantly enhance predictive performance for peritoneal metastasis in CRC. Visceral fat is a meaningful imaging biomarker for peritoneal metastasis's early detection in CRC patients. KEY POINTS: Question Current research on predicting colorectal cancer with peritoneal metastasis mainly focuses on single-modality analysis, while studies based on multimodal imaging information are relatively scarce. Findings The Multi-scale Feature Fusion Network, constructed based on ResNet18, can utilize CT images of tumors and visceral fat to detect occult peritoneal metastasis in colorectal cancer. Clinical relevance This study identified independent prognostic factors for colorectal cancer peritoneal metastasis and combines them with tumor and visceral fat network features, aiding early diagnosis and accurate prognostic assessment.