External validation of a motion capture-based surgical skill assessment system in laparoscopic simulation training environments.

purposeTo externally validate our surgical skill assessment system, which provides comprehensive real-time feedback based on motion capture (Mocap) metrics of laparoscopic instruments in simulation training environments.

methodsParticipants conducted periaortic tissue dissection (Dissection Task) and parenchymal closure of the kidney (Suturing Task) using porcine cadaveric organs. Surgical instrument movements were measured by the MoCap system, and all training sessions were video-recorded. Utilizing our previously developed machine-learning (ML) algorithm, participants received quantitative feedback based on their motion features immediately after the training, including three-tier classification results, which meant their skill levels being consistent with those with

resultsA total of 38 urologists, 4 junior residents, and 10 medical students voluntarily participated in the current training. Several participants repeatedly performed the training, and so 107 datasets were included in the present analysis. Validation analyses demonstrated overall classification accuracies of 67.3 and 56.9% for Dissection Task and Suturing Task, respectively, when excluding the subsequent data of repeaters. In terms of GOALS score prediction, the correlation coefficient and MAE were 0.86 and 2.51

conclusionsOur real-time feedback system was viewed favorably by the participants, and the ML algorithm was a reliable method, especially the predicted GOALS score. This system could facilitate objective performance assessment, enhance simulation-based training efficacy, and potentially accelerate surgical skill acquisition.