Vision-Based Dynamic Virtual Fixtures for Tools Collision Avoidance in Robotic Surgery

In robot-aided surgery, during the execution of typical bimanual procedures such as dissection, surgical tools can collide and create serious damage to the robot or tissues. The da Vinci robot is one of the most advanced and certainly the most widespread robotic system dedicated to minimally invasive surgery. Although the procedures performed by da Vinci-like surgical robots are teleoperated, potential collisions between surgical tools are a very sensitive issue declared by surgeons. Shared control techniques based on Virtual Fixtures (VF) can be an effective way to help the surgeon prevent tools collision. This letter presents a surgical tools collision avoidance method that uses Forbidden Region Virtual Fixtures. Tool clashing is avoided by rendering a repulsive force to the surgeon. To ensure the correct definition of the VF, a marker-less tool tracking method, using deep neural network architecture for tool segmentation, is adopted. The use of direct kinematics for tools collision avoidance is affected by tools position error introduced by robot component elasticity during tools interaction with the environment. On the other hand, kinematics information can help in case of occlusions of the camera. Therefore, this work proposes an Extended Kalman Filter (EKF) for pose estimation which ensures a more robust application of VF on the tool, coupling vision and kinematics information. The entire pipeline is tested in different tasks using the da Vinci Research Kit system.