A New Tensioning Method using Deep Reinforcement Learning for Surgical Pattern Cutting

Surgeons normally need surgical scissors and tissue grippers to cut through a deformable surgical tissue. The cutting accuracy depends on the skills to manipulate these two tools. Such skills are part of basic surgical skills training as in the Fundamentals of Laparoscopic Surgery. The gripper is used to pinch a point on the surgical sheet and pull the tissue to a certain direction to maintain the tension while the scissors cut through a trajectory. As the surgical materials are deformable, it requires a comprehensive tensioning policy to yield appropriate tensioning direction at each step of the cutting process. Automating a tensioning policy for a given cutting trajectory will support not only the human surgeons but also the surgical robots to improve the cutting accuracy and reliability. This paper presents a multiple pinch point approach to modelling an autonomous tensioning planner based on a deep reinforcement learning algorithm. Experiments on a simulator show that the proposed method is superior to existing methods in terms of both performance and robustness.

[1]  Pieter Abbeel,et al.  Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations , 2010, 2010 IEEE International Conference on Robotics and Automation.

[2]  Pieter Abbeel,et al.  Benchmarking Deep Reinforcement Learning for Continuous Control , 2016, ICML.

[3]  J. Kaouk,et al.  Fundamental skills of robotic surgery: a multi-institutional randomized controlled trial for validation of a simulation-based curriculum. , 2013, Urology.

[4]  Saeid Nahavandi,et al.  Seeded transfer learning for regression problems with deep learning , 2019, Expert Syst. Appl..

[5]  Mark E Rentschler,et al.  Towards autonomous motion control in minimally invasive robotic surgery , 2016, Expert review of medical devices.

[6]  Arianna Menciassi,et al.  Design and development of a soft robotic gripper for manipulation in minimally invasive surgery: a proof of concept , 2015 .

[7]  Wei Wang,et al.  Variable Admittance Control Based on Fuzzy Reinforcement Learning for Minimally Invasive Surgery Manipulator , 2017, Sensors.

[8]  Allison M. Okamura,et al.  Methods to Segment Hard Inclusions in Soft Tissue During Autonomous Robotic Palpation , 2015, IEEE Transactions on Robotics.

[9]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[10]  Alois Knoll,et al.  Automation of tissue piercing using circular needles and vision guidance for computer aided laparoscopic surgery , 2010, 2010 IEEE International Conference on Robotics and Automation.

[11]  Wenjun Xu,et al.  Towards transferring skills to flexible surgical robots with programming by demonstration and reinforcement learning , 2016, 2016 Eighth International Conference on Advanced Computational Intelligence (ICACI).

[12]  K. M. Deliparaschos,et al.  Evolution of autonomous and semi‐autonomous robotic surgical systems: a review of the literature , 2011, The international journal of medical robotics + computer assisted surgery : MRCAS.

[13]  Dong-Soo Kwon,et al.  Path Planning for Automation of Surgery Robot based on Probabilistic Roadmap and Reinforcement Learning , 2018, 2018 15th International Conference on Ubiquitous Robots (UR).

[14]  Christian Duriez,et al.  Real-time simulation of contact and cutting of heterogeneous soft-tissues , 2014, Medical Image Anal..

[15]  Murray Campbell,et al.  Singular Extensions: Adding Selectivity to Brute-Force Searching , 1990, Artif. Intell..

[16]  Ole Vegard Solberg,et al.  Navigated laparoscopic ultrasound in abdominal soft tissue surgery: technological overview and perspectives , 2012, International Journal of Computer Assisted Radiology and Surgery.

[17]  Kenneth Y. Goldberg,et al.  Automating multi-throw multilateral surgical suturing with a mechanical needle guide and sequential convex optimization , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Pieter Abbeel,et al.  Learning by observation for surgical subtasks: Multilateral cutting of 3D viscoelastic and 2D Orthotropic Tissue Phantoms , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Allison M. Okamura,et al.  A paced shared-control teleoperated architecture for supervised automation of multilateral surgical tasks , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[20]  Thanh Thi Nguyen,et al.  A Multi-Objective Deep Reinforcement Learning Framework , 2018, Eng. Appl. Artif. Intell..

[21]  Saeid Nahavandi,et al.  A sequential search-space shrinking using CNN transfer learning and a Radon projection pool for medical image retrieval , 2018, Expert Syst. Appl..

[22]  Daniel J Scott,et al.  Design of a Proficiency-Based Skills Training Curriculum for the Fundamentals of Laparoscopic Surgery , 2007, Surgical innovation.

[23]  Allison M. Okamura,et al.  Modeling the Forces of Cutting With Scissors , 2008, IEEE Transactions on Biomedical Engineering.

[24]  Saeid Nahavandi,et al.  System Design Perspective for Human-Level Agents Using Deep Reinforcement Learning: A Survey , 2017, IEEE Access.

[25]  Mohammad Biglarbegian,et al.  State of the Art Robotic Grippers and Applications , 2016, Robotics.

[26]  Mamoru Mitsuishi,et al.  Online Trajectory Planning and Force Control for Automation of Surgical Tasks , 2018, IEEE Transactions on Automation Science and Engineering.

[27]  Danail Stoyanov,et al.  Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions , 2016, International Journal of Computer Assisted Radiology and Surgery.

[28]  Nazim Haouchine,et al.  Impact of Soft Tissue Heterogeneity on Augmented Reality for Liver Surgery , 2015, IEEE Transactions on Visualization and Computer Graphics.

[29]  Ryan S. Decker,et al.  Supervised autonomous robotic soft tissue surgery , 2016, Science Translational Medicine.

[30]  Brijen Thananjeyan,et al.  Multilateral surgical pattern cutting in 2D orthotropic gauze with deep reinforcement learning policies for tensioning , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[31]  Saeid Nahavandi,et al.  Multi-Agent Deep Reinforcement Learning with Human Strategies , 2018, 2019 IEEE International Conference on Industrial Technology (ICIT).

[32]  Nabeel A. Arain,et al.  Developing a comprehensive, proficiency-based training program for robotic surgery. , 2012, Surgery.

[33]  Chee Peng Lim,et al.  Medical image analysis using wavelet transform and deep belief networks , 2017, Expert Syst. Appl..

[34]  Prokar Dasgupta,et al.  An over-view of robot assisted surgery curricula and the status of their validation. , 2015, International journal of surgery.

[35]  Stuart M. Gale,et al.  Patterning of tensile fabric structures with a discrete element model using dynamic relaxation , 2016 .

[36]  Saeid Nahavandi,et al.  A Human Mixed Strategy Approach to Deep Reinforcement Learning , 2018, 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC).