Fast and Reliable Autonomous Surgical Debridement with Cable-Driven Robots Using a Two-Phase Calibration Procedure

Automating precision subtasks such as debridement (removing dead or diseased tissue fragments) with Robotic Surgical Assistants (RSAs) such as the da Vinci Research Kit (dVRK) is challenging due to inherent nOnlinearities in cable-driven systems. We propose and evaluate a novel two-phase coarse-to-fine calibration method. In Phase I (coarse), we place a red calibration marker on the end effector and let it randomly move through a set of open-loop trajectories to obtain a large sample set of camera pixels and internal robot end-effector configurations. This coarse data is then used to train a Deep Neural Network (DNN) to learn the coarse transformation bias. In Phase II (fine), the bias from Phase I is applied to move the end -effector toward a small set of specific target points on a printed sheet. For each target, a human operator manually adjusts the end -effector position by direct contact (not through teleoperation) and the residual compensation bias is recorded. This fine data is then used to train a Random Forest (RF) to learn the fine transformation bias. Subsequent experiments suggest that without calibration, position errors average 4.55mm. Phase I can reduce average error to 2.14mm and the combination of Phase I and Phase II can reduces average error to 1.08mm. We apply these results to debridement of raisins and pumpkin seeds as fragment phantoms. Using an endoscopic stereo camera with standard edge detection, experiments with 120 trials achieved average success rates of 94.5 %, exceeding prior results with much larger fragments (89.4%) and achieving a speedup of 2.1x, decreasing time per fragment from 15.8 seconds to 7.3 seconds. Source code, data, and videos are available at https://sites.google.com/view/calib-icra/.

[1]  John M. Hollerbach,et al.  Autonomous Robot Calibration for Hand-Eye Coordination , 1991, Int. J. Robotics Res..

[2]  C. Attinger,et al.  Surgical débridement. The key to successful wound healing and reconstruction. , 2000, Clinics in podiatric medicine and surgery.

[3]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[4]  Danica Kragic,et al.  Survey on Visual Servoing for Manipulation , 2002 .

[5]  Garth H Ballantyne,et al.  The da Vinci telerobotic surgical system: the virtual operative field and telepresence surgery. , 2003, The Surgical clinics of North America.

[6]  Allison M. Okamura,et al.  Force modeling for needle insertion into soft tissue , 2004, IEEE Transactions on Biomedical Engineering.

[7]  M. Granick,et al.  Toward a common language: surgical wound bed preparation and debridement , 2006, Wound repair and regeneration : official publication of the Wound Healing Society [and] the European Tissue Repair Society.

[8]  Wisama Khalil,et al.  Model Identification , 2019, Springer Handbook of Robotics, 2nd Ed..

[9]  Quoc V. Le,et al.  Joint calibration of multiple sensors , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Pieter Abbeel,et al.  Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations , 2010, 2010 IEEE International Conference on Robotics and Automation.

[11]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[12]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[13]  K. M. Deliparaschos,et al.  Evolution of autonomous and semi‐autonomous robotic surgical systems: a review of the literature , 2011, The international journal of medical robotics + computer assisted surgery : MRCAS.

[14]  Ryan A. Beasley Medical Robots: Current Systems and Research Directions , 2012, J. Robotics.

[15]  Gaurav S. Sukhatme,et al.  Learning task error models for manipulation , 2013, 2013 IEEE International Conference on Robotics and Automation.

[16]  Blake Hannaford,et al.  Raven-II: An Open Platform for Surgical Robotics Research , 2013, IEEE Transactions on Biomedical Engineering.

[17]  Ankush Gupta,et al.  A case study of trajectory transfer through non-rigid registration for a simplified suturing scenario , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Peter Kazanzides,et al.  An open-source research kit for the da Vinci® Surgical System , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Pieter Abbeel,et al.  Learning accurate kinematic control of cable-driven surgical robots using data cleaning and Gaussian Process Regression , 2014, 2014 IEEE International Conference on Automation Science and Engineering (CASE).

[20]  Mehran Anvari,et al.  The Future of Robotic Platforms , 2014 .

[21]  Pieter Abbeel,et al.  Autonomous multilateral debridement with the Raven surgical robot , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Allison M. Okamura,et al.  Models of human-centered automation in a debridement task , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  Danail Stoyanov,et al.  Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions , 2016, International Journal of Computer Assisted Radiology and Surgery.

[24]  Olivier Lambercy,et al.  Motion planning for a multi-arm surgical robot using both sampling-based algorithms and motion primitives , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[25]  Pieter Abbeel,et al.  Learning by observation for surgical subtasks: Multilateral cutting of 3D viscoelastic and 2D Orthotropic Tissue Phantoms , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[27]  Blake Hannaford,et al.  Measurement of the cable-pulley Coulomb and viscous friction for a cable-driven surgical robotic system , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[28]  Blake Hannaford,et al.  Unscented Kalman Filter and 3D vision to improve cable driven surgical robot joint angle estimation , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[29]  Trevor Darrell,et al.  TSC-DL: Unsupervised trajectory segmentation of multi-modal surgical demonstrations with Deep Learning , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[30]  Kenneth Y. Goldberg,et al.  Automating multi-throw multilateral surgical suturing with a mechanical needle guide and sequential convex optimization , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[31]  Kenneth Y. Goldberg,et al.  Tumor localization using automated palpation with Gaussian Process Adaptive Sampling , 2016, 2016 IEEE International Conference on Automation Science and Engineering (CASE).

[32]  Brijen Thananjeyan,et al.  Multilateral surgical pattern cutting in 2D orthotropic gauze with deep reinforcement learning policies for tensioning , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[33]  Sergey Levine,et al.  Learning Visual Servoing with Deep Features and Fitted Q-Iteration , 2017, ICLR.

[34]  Ion Stoica,et al.  DDCO: Discovery of Deep Continuous Options for Robot Learning from Demonstrations , 2017, CoRL.

[35]  Michael C. Yip,et al.  Robot Autonomy for Surgery , 2017, The Encyclopedia of Medical Robotics.

[36]  Sergey Levine,et al.  Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection , 2016, Int. J. Robotics Res..