Learning by observation for surgical subtasks: Multilateral cutting of 3D viscoelastic and 2D Orthotropic Tissue Phantoms

Automating repetitive surgical subtasks such as suturing, cutting and debridement can reduce surgeon fatigue and procedure times and facilitate supervised tele-surgery. Programming is difficult because human tissue is deformable and highly specular. Using the da Vinci Research Kit (DVRK) robotic surgical assistant, we explore a “Learning By Observation” (LBO) approach where we identify, segment, and parameterize motion sequences and sensor conditions to build a finite state machine (FSM) for each subtask. The robot then executes the FSM repeatedly to tune parameters and if necessary update the FSM structure. We evaluate the approach on two surgical subtasks: debridement of 3D Viscoelastic Tissue Phantoms (3d-DVTP), in which small target fragments are removed from a 3D viscoelastic tissue phantom; and Pattern Cutting of 2D Orthotropic Tissue Phantoms (2d-PCOTP), a step in the standard Fundamentals of Laparoscopic Surgery training suite, in which a specified circular area must be cut from a sheet of orthotropic tissue phantom. We describe the approach and physical experiments with repeatability of 96% for 50 trials of the 3d-DVTP subtask and 70% for 20 trials of the 2d-PCOTP subtask. A video is available at: http://j.mp/Robot-Surgery-Video-Oct-2014.

[1]  C. Attinger,et al.  Surgical débridement. The key to successful wound healing and reconstruction. , 2000, Clinics in podiatric medicine and surgery.

[2]  Han-Wen Nienhuys,et al.  A Surgery Simulation Supporting Cuts and Finite Element Deformation , 2001, MICCAI.

[3]  W. Boyd,et al.  Hybrid robotic coronary artery surgery and angioplasty in multivessel coronary artery disease. , 2002, The Annals of thoracic surgery.

[4]  Russell H. Taylor,et al.  Medical robotics in computer-integrated surgery , 2003, IEEE Trans. Robotics Autom..

[5]  Garth H Ballantyne,et al.  The da Vinci telerobotic surgical system: the virtual operative field and telepresence surgery. , 2003, The Surgical clinics of North America.

[6]  Pradeep K. Khosla,et al.  Inferring user intent for learning by observation , 2004 .

[7]  Sir Ara W Darzi,et al.  The impact of minimally invasive surgical techniques. , 2004, Annual review of medicine.

[8]  Hui Zhang,et al.  On cutting and dissection of virtual deformable objects , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[9]  R. Veldkamp,et al.  Laparoscopic surgery versus open surgery for colon cancer : short-term outcomes of a randomised trial , 2022 .

[10]  Ph.D. Susan M.L. Lim FRCS,et al.  Nerve-sparing Axillary Dissection Using the da Vinci Surgical System , 2005, World Journal of Surgery.

[11]  M. Granick,et al.  Toward a common language: surgical wound bed preparation and debridement , 2006, Wound repair and regeneration : official publication of the Wound Healing Society [and] the European Tissue Repair Society.

[12]  Gregory D. Hager,et al.  Towards automatic skill evaluation: detection and segmentation of robot-assisted surgical motions. , 2006 .

[13]  Daniel J Scott,et al.  Design of a Proficiency-Based Skills Training Curriculum for the Fundamentals of Laparoscopic Surgery , 2007, Surgical innovation.

[14]  Kenneth Y. Goldberg,et al.  Motion Planning in Medicine: Optimization and Simulation Algorithms for Image-Guided Procedures , 2008, Springer Tracts in Advanced Robotics.

[15]  Darius Burschka,et al.  Automation of Manual Tasks for Minimally Invasive Surgery , 2008, Fourth International Conference on Autonomic and Autonomous Systems (ICAS'08).

[16]  Gary R. Bradski,et al.  Learning OpenCV - computer vision with the OpenCV library: software that sees , 2008 .

[17]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[18]  Russell H. Taylor,et al.  Medical robotics in computer-integrated surgery , 2003, IEEE Trans. Robotics Autom..

[19]  Gregory D. Hager,et al.  Data-Derived Models for Segmentation with Application to Surgical Assessment and Training , 2009, MICCAI.

[20]  James F. O'Brien,et al.  Interactive simulation of surgical needle insertion and steering , 2009, ACM Trans. Graph..

[21]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[22]  Alon Wolf,et al.  Medical Automation and Robotics , 2009, Handbook of Automation.

[23]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[24]  Gregory D. Hager,et al.  Task versus Subtask Surgical Skill Evaluation of Robotic Minimally Invasive Surgery , 2009, MICCAI.

[25]  Pieter Abbeel,et al.  Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations , 2010, 2010 IEEE International Conference on Robotics and Automation.

[26]  Jochen J. Steil,et al.  Imitating object movement skills with robots — A task-level approach exploiting generalization and invariance , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  Gregory D. Hager,et al.  Motion generation of robotic surgical tasks: Learning from expert demonstrations , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[28]  Gregory D. Hager,et al.  Human-Machine Collaborative surgery using learned models , 2011, 2011 IEEE International Conference on Robotics and Automation.

[29]  K. M. Deliparaschos,et al.  Evolution of autonomous and semi‐autonomous robotic surgical systems: a review of the literature , 2011, The international journal of medical robotics + computer assisted surgery : MRCAS.

[30]  Nabeel A. Arain,et al.  Developing a comprehensive, proficiency-based training program for robotic surgery. , 2012, Surgery.

[31]  Ryan A. Beasley.  Medical Robots: Current Systems and Research Directions , 2012, J. Robotics.

[32]  Scott Kuindersma,et al.  Robot learning from demonstration by constructing skill trees , 2012, Int. J. Robotics Res..

[33]  Russell H. Taylor,et al.  System for robot-assisted real-time laparoscopic ultrasound elastography , 2012, Medical Imaging.

[34]  Blake Hannaford,et al.  Raven-II: An Open Platform for Surgical Robotics Research , 2013, IEEE Transactions on Biomedical Engineering.

[35]  Ankush Gupta,et al.  A case study of trajectory transfer through non-rigid registration for a simplified suturing scenario , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[36]  Peter Kazanzides,et al.  An open-source research kit for the da Vinci® Surgical System , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[37]  Mamoru Mitsuishi,et al.  Online Trajectory Planning in Dynamic Environments for Surgical Task Automation , 2014, Robotics: Science and Systems.

[38]  Pieter Abbeel,et al.  Autonomous multilateral debridement with the Raven surgical robot , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[39]  Scott Niekum,et al.  Learning grounded finite-state representations from unstructured demonstrations , 2015, Int. J. Robotics Res..

[40]  Adrian Kaehler,et al.  Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library , 2016 .