Manipulating Soft Tissues by Deep Reinforcement Learning for Autonomous Robotic Surgery

In robotic surgery, pattern cutting through a deformable material is a challenging research field. The cutting procedure requires a robot to concurrently manipulate a scissor and a gripper to cut through a predefined contour trajectory on the deformable sheet. The gripper ensures the cutting accuracy by nailing a point on the sheet and continuously tensioning the pinch point to different directions while the scissor is in action. The goal is to find a pinch point and a corresponding tensioning policy to minimize damage to the material and increase cutting accuracy measured by the symmetric difference between the predefined contour and the cut contour. Previous study considers finding one fixed pinch point during the course of cutting, which is inaccurate and unsafe when the contour trajectory is complex. In this paper, we examine the soft tissue cutting task by using multiple pinch points, which imitates human operations while cutting. This approach, however, does not require the use of a multi-gripper robot. We use a deep reinforcement learning algorithm to find an optimal tensioning policy of a pinch point. Simulation results show that the multi-point approach outperforms the state-of the-art method in soft pattern cutting task with respect to both accuracy and reliability.

[1]  Saeid Nahavandi,et al.  Multi-Agent Deep Reinforcement Learning with Human Strategies , 2018, 2019 IEEE International Conference on Industrial Technology (ICIT).

[2]  Saeid Nahavandi,et al.  Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications , 2018, IEEE Transactions on Cybernetics.

[3]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[4]  Brijen Thananjeyan,et al.  Multilateral surgical pattern cutting in 2D orthotropic gauze with deep reinforcement learning policies for tensioning , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[5]  Allison M. Okamura,et al.  Methods to Segment Hard Inclusions in Soft Tissue During Autonomous Robotic Palpation , 2015, IEEE Transactions on Robotics.

[6]  Nabeel A. Arain,et al.  Developing a comprehensive, proficiency-based training program for robotic surgery. , 2012, Surgery.

[7]  K. M. Deliparaschos,et al.  Evolution of autonomous and semi‐autonomous robotic surgical systems: a review of the literature , 2011, The international journal of medical robotics + computer assisted surgery : MRCAS.

[8]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[9]  Oliver Kroemer,et al.  Learning to select and generalize striking movements in robot table tennis , 2012, AAAI Fall Symposium: Robots Learning Interactively from Human Teachers.

[10]  Arianna Menciassi,et al.  Design and development of a soft robotic gripper for manipulation in minimally invasive surgery: a proof of concept , 2015 .

[11]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[12]  Pieter Abbeel,et al.  Learning by observation for surgical subtasks: Multilateral cutting of 3D viscoelastic and 2D Orthotropic Tissue Phantoms , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Martin A. Riedmiller,et al.  Reinforcement learning for robot soccer , 2009, Auton. Robots.

[14]  Thanh Thi Nguyen,et al.  A Multi-Objective Deep Reinforcement Learning Framework , 2018, Eng. Appl. Artif. Intell..

[15]  Saeid Nahavandi,et al.  A New Tensioning Method using Deep Reinforcement Learning for Surgical Pattern Cutting , 2019, 2019 IEEE International Conference on Industrial Technology (ICIT).

[16]  Henk Nijmeijer,et al.  Robot Programming by Demonstration , 2010, SIMPAR.

[17]  Srikanth V. Krishnamurthy,et al.  IotSan: fortifying the safety of IoT systems , 2018, CoNEXT.

[18]  Mamoru Mitsuishi,et al.  Online Trajectory Planning and Force Control for Automation of Surgical Tasks , 2018, IEEE Transactions on Automation Science and Engineering.

[19]  Kenneth Y. Goldberg,et al.  Automating multi-throw multilateral surgical suturing with a mechanical needle guide and sequential convex optimization , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Stefan Schaal,et al.  Learning from Demonstration , 1996, NIPS.

[21]  Ben Tse,et al.  Autonomous Inverted Helicopter Flight via Reinforcement Learning , 2004, ISER.

[22]  Nazim Haouchine,et al.  Impact of Soft Tissue Heterogeneity on Augmented Reality for Liver Surgery , 2015, IEEE Transactions on Visualization and Computer Graphics.

[23]  Ryan S. Decker,et al.  Supervised autonomous robotic soft tissue surgery , 2016, Science Translational Medicine.

[24]  Saeid Nahavandi,et al.  System Design Perspective for Human-Level Agents Using Deep Reinforcement Learning: A Survey , 2017, IEEE Access.

[25]  Mohammad Biglarbegian,et al.  State of the Art Robotic Grippers and Applications , 2016, Robotics.

[26]  Wei Wang,et al.  Variable Admittance Control Based on Fuzzy Reinforcement Learning for Minimally Invasive Surgery Manipulator , 2017, Sensors.

[27]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[28]  Alois Knoll,et al.  Automation of tissue piercing using circular needles and vision guidance for computer aided laparoscopic surgery , 2010, 2010 IEEE International Conference on Robotics and Automation.

[29]  Wenjun Xu,et al.  Towards transferring skills to flexible surgical robots with programming by demonstration and reinforcement learning , 2016, 2016 Eighth International Conference on Advanced Computational Intelligence (ICACI).

[30]  Danail Stoyanov,et al.  Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions , 2016, International Journal of Computer Assisted Radiology and Surgery.

[31]  Saeid Nahavandi,et al.  A Human Mixed Strategy Approach to Deep Reinforcement Learning , 2018, 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[32]  Prokar Dasgupta,et al.  An over-view of robot assisted surgery curricula and the status of their validation. , 2015, International journal of surgery.

[33]  Ankush Gupta,et al.  A case study of trajectory transfer through non-rigid registration for a simplified suturing scenario , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[34]  Peter Kazanzides,et al.  An open-source research kit for the da Vinci® Surgical System , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[35]  Mark E Rentschler,et al.  Towards autonomous motion control in minimally invasive robotic surgery , 2016, Expert review of medical devices.

[36]  Han-Wen Nienhuys,et al.  A Surgery Simulation Supporting Cuts and Finite Element Deformation , 2001, MICCAI.

[37]  Pieter Abbeel,et al.  Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations , 2010, 2010 IEEE International Conference on Robotics and Automation.

[38]  Pieter Abbeel,et al.  Benchmarking Deep Reinforcement Learning for Continuous Control , 2016, ICML.

[39]  J. Kaouk,et al.  Fundamental skills of robotic surgery: a multi-institutional randomized controlled trial for validation of a simulation-based curriculum. , 2013, Urology.