Haptic-guided shared control for needle grasping optimization in minimally invasive robotic surgery

During suturing tasks performed with minimally invasive surgical robots, configuration singularities and joint limits often force surgeons to interrupt the task and re-grasp the needle using dual-arm movements. This yields an increased operator’s cognitive load, time-to-completion and performance degradation. In this paper, we propose a haptic-guided shared control method for grasping the needle with the Patient Side Manipulator (PSM) of the da Vinci robot avoiding such issues. We suggest a cost function consisting of (i) the distance from robot joint limits and (ii) the task-oriented manipulability along the suturing trajectory. Evaluating the cost and its gradient on the needle grasping manifold allows us to obtain the optimal grasping pose for joint-limit and singularity free robot movements during suturing. We compute force cues and display them through the Master Tool Manipulator (MTM) to guide the surgeon towards the optimal grasp. As such, our system helps the operator to choose a grasping configuration that allows the robot to avoid joint limits and singularities during post-grasp suturing movements. We show the effectiveness of the proposed haptic-guided shared control method during suturing using both simulated and real experiments. The results illustrate that our approach significantly improves the performance in terms of needle re-grasping.

[1]  Castellani Marco,et al.  Analysis of the inertia and dynamics of grasped objects, for choosing optimal grasps to enable torque-efficient post-grasp manipulations , 2016 .

[2]  Philippe Zanne,et al.  Stitching Planning in Laparoscopic Surgery: Towards Robot-assisted Suturing , 2009, Int. J. Robotics Res..

[3]  Kate Saenko,et al.  Grasp Pose Detection in Point Clouds , 2017, Int. J. Robotics Res..

[4]  Alessandro De Luca,et al.  Sensorless Robot Collision Detection and Hybrid Force/Motion Control , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[5]  Elena De Momi,et al.  Automated Pick-Up of Suturing Needles for Robotic Surgical Assistance , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Claudio Pacchierotti,et al.  Haptic-Based Shared-Control Methods for a Dual-Arm System , 2018, IEEE Robotics and Automation Letters.

[7]  Paolo Robuffo Giordano,et al.  Human-in-the-loop optimisation: Mixed initiative grasping for optimally facilitating post-grasp manipulative actions , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[8]  Murat Cenk Cavusoglu,et al.  Needle Grasp and Entry Port Selection for Automatic Execution of Suturing Tasks in Robotic Minimally Invasive Surgery , 2016, IEEE Transactions on Automation Science and Engineering.

[9]  Bruno Siciliano,et al.  Modelling and identification of the da Vinci Research Kit robotic arms , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[10]  Larry H. Matthies,et al.  Task-oriented grasping with semantic and geometric scene understanding , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  Mamoru Mitsuishi,et al.  Online Trajectory Planning in Dynamic Environments for Surgical Task Automation , 2014, Robotics: Science and Systems.

[12]  Mamoru Mitsuishi,et al.  Active Constraints Using Vector Field Inequalities for Surgical Robots , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[13]  E. AmirM.Ghalamzan,et al.  Choosing Grasps to Enable Collision-Free Post-Grasp Manipulations , 2018, 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids).

[14]  Ales Leonardis,et al.  One-shot learning and generation of dexterous grasps for novel objects , 2016, Int. J. Robotics Res..

[15]  Murat Cenk Cavusoglu,et al.  Needle path planning for autonomous robotic surgical suturing , 2013, 2013 IEEE International Conference on Robotics and Automation.

[16]  Bruno Siciliano,et al.  Passive Virtual Fixtures Adaptation in Minimally Invasive Robotic Surgery , 2018, IEEE Robotics and Automation Letters.

[17]  Bruno Siciliano,et al.  A V-REP Simulator for the da Vinci Research Kit Robotic Platform , 2018, 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob).

[18]  Bruno Siciliano,et al.  A New Laparoscopic Tool With In-Hand Rolling Capabilities for Needle Reorientation , 2018, IEEE Robotics and Automation Letters.

[19]  Kenneth Y. Goldberg,et al.  Automating multi-throw multilateral surgical suturing with a mechanical needle guide and sequential convex optimization , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Jun Nakanishi,et al.  Learning Movement Primitives , 2005, ISRR.

[21]  Peter Kazanzides,et al.  An open-source research kit for the da Vinci® Surgical System , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Gregory D. Hager,et al.  A Dataset and Benchmarks for Segmentation and Recognition of Gestures in Robotic Surgery , 2017, IEEE Transactions on Biomedical Engineering.

[23]  E. AmirM.Ghalamzan,et al.  Robot learning from demonstrations: Emulation learning in environments with moving obstacles , 2018, Robotics Auton. Syst..

[24]  Kate Saenko,et al.  High precision grasp pose detection in dense clutter , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[25]  Murat Cenk Cavusoglu,et al.  Optimal needle grasp selection for automatic execution of suturing tasks in robotic minimally invasive surgery , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).