Deep reinforcement learning in continuous action space for autonomous robotic surgery.

purposeReinforcement learning methods have shown promising results for the automation of sub-tasks in robotic surgery systems. With the development of these methods, surgical robots have been able to achieve good performances, so that they can be used in complex and high-risk environments such as surgical pattern cutting to reduce stress and pressure on the surgeon and increase surgical accuracy. This study has aimed at providing a deep reinforcement learning-based approach to control the gripper arm when cutting soft tissue in a continuous action space.

methodsSurgical soft tissue cutting in this study is performed by controlling the gripper arm in a continuous action space and a grid observation space. In the proposed method using deep reinforcement learning, we find an optimal tensioning policy in the continuous action space that increases the cutting accuracy of the predetermined pattern.

resultsThe simulation results demonstrated that in the cutting of many complex patterns, the proposed method works better than the methods in which the tensioning was performed in a discrete action space and the observation space was modeled as a partial and random representation.

conclusionWe introduced a deep reinforcement learning-based method for obtaining the optimal tensioning policy in a continuous action space when cutting a predetermined pattern. We showed that the proposed approach outperforms the state-of-the-art method in the soft pattern cutting task with respect to accuracy.