Privacy-Preserving Surgical Video Analysis with Swarm Learning - Results from a Multinational Appendectomy Cohort.

backgroundProgress in artificial intelligence (AI)-based analysis of surgical videos has been constrained by reliance on manual frame-level annotations rather than patient-level outcomes. In addition, concerns about data privacy restrict the exchange of laparoscopic video data and, thereby, multicenter collaboration.

methodsTo address these limitations, we developed a pipeline that integrates weakly supervised deep learning with Swarm Learning, a decentralized machine learning approach that enables collaborative model training without data centralization. We evaluated our pipeline using a dataset of 397 laparoscopic appendectomy recordings from six international centers for two patient-level staging tasks: (1) laparoscopic grading of appendicitis and appendiceal perforation detection; and (2) histopathologic inflammation grading. We identified optimal modeling configurations (frame sampling rates and model architectures) using the binary perforation detection task, then compared Swarm Learning with single-center and centralized learning across the laparoscopic and histopathologic disease staging tasks. In addition, we surveyed participating centers to identify barriers to clinical implementation of our learning pipeline for surgical video analysis.

resultsFor binary perforation detection, frame sampling at one frame per second and use of the SurgTempoNet architecture resulted in reliable classification performance, outperforming SurgFrameNet and Multiple Instance Learning. For both laparoscopic (area under the receiver operating curve [AUROC]: 0.818±0.092) and histopathologic disease staging (AUROC: 0.626±0.029), Swarm Learning consistently outperformed single-center training and achieved performance comparable to centralized learning on external validation (AUROC: 0.795±0.092 for laparoscopic grading; AUROC: 0.610±0.018 for histopathologic grading). The user survey identified hardware failure and limited integration of the decentralized learning pipeline with electronic patient records as key barriers to clinical implementation.

conclusionsWeakly supervised deep learning enables the prediction of patient-level labels directly from surgical video data. Swarm Learning facilitates privacy-preserving multicenter collaboration and achieves performance on par with centralized learning, highlighting its potential for advancing clinically relevant, collaborative AI development in surgical video analysis. (Funded by the European Union and others.).