A Novel Switching of Artificial Intelligence to Generate Simultaneously Multimodal Images to Assess Inflammation and Predict Outcomes in Ulcerative Colitis-(With Video).

objectivesVirtual Chromoendoscopy (VCE) is pivotal for assessing activity and predicting outcomes in Ulcerative Colitis (UC), though interobserver variability and the need for expertise persist. Artificial intelligence (AI) offers standardized VCE-based assessment. This study introduces a novel AI model to detect and simultaneously generate various endoscopic modalities, enhancing AI-driven inflammation assessment and outcome prediction in UC.

methodsEndoscopic videos in high-definition white-light, iScan2, iScan3, and NBI from UC patients of the international PICaSSO iScan and NBI cohort (302 and 54 patients, respectively) were used to develop a neural network to identify the acquisition modality of each frame and for inter-modality image switching. 2535 frames from 169 videos of the iScan cohort were switched to different modalities and trained a deep-learning model for inflammation assessment. Subsequently, the model was tested on a subset of the iScan and NBI cohorts (72 and 51 videos, respectively). Performance in predicting endoscopic and histological activity and outcomes was evaluated.

resultsThe model efficiently classified and converted images across modalities (92% accuracy). Performance in predicting endoscopic and histological remission was excellent, especially with different modalities combined in both iScan (accuracy 81.3% and 89.6%; AUROC 0.92 and 0.89 by UCEIS and PICaSSO, respectively) and the NBI cohort. Moreover, it showed a remarkable ability in predicting clinical outcomes.

conclusionsOur multimodal "AI-switching" model innovatively detects and transitions between different endoscopic modalities, refining inflammation assessment and outcome prediction in UC by integrating model-derived images.