St-Swin TransNet: a spatiotemporal swin transformer-based network for self-supervised depth estimation in stereoscopic surgical videos.

purposeDepth estimation from stereoscopic laparoscopic videos is of vital importance in computer-assisted intervention due to its potential for downstream tasks in laparoscopic surgical navigation. Previous works mostly focus on depth estimation from static frames, while temporal information in stereoscopic laparoscopic videos is largely ignored.

methodsA spatiotemporal swin (ST-Swin) transformer-based network, referred to as ST-Swin TransNet, is proposed for depth estimation in stereoscopic surgical videos. Built upon a symmetric encoder-decoder architecture consisting of 12 ST-Swin blocks, ST-Swin TransNet extracts spatiotemporal features for efficient and accurate depth estimation, where the ST-Swin blocks are designed to capture spatiotemporal information from stereo video sequences via self-attention mechanism. Given binocular laparoscopic videos, ST-Swin TransNet exploits hierarchical spatiotemporal features to predict disparity maps.

resultsComprehensive experiments are conducted on two typical yet challenging public datasets to evaluate the performance of the proposed method. We additionally demonstrate the feasibility of applying ST-Swin TransNet to video see-through augmented reality (VST-AR) navigation in laparoscopic surgery. Our method achieved a mean absolute depth error (mADE) of 3.33 mm in depth estimation and a mean absolute distance (mAD) of 1.07 mm in VST-AR navigation.

conclusionA spatiotemporal swin transformer-based network for self-supervised depth estimation in binocular laparoscopic surgical videos was developed. Results from the comprehensive experiments demonstrate the superior performance of the proposed method over the state-of-the-art methods.