Automated Pick-Up of Suturing Needles for Robotic Surgical Assistance

Robot-assisted laparoscopic prostatectomy (RALP) is a treatment for prostate cancer that involves complete or nerve sparing removal prostate tissue that contains cancer. After removal the bladder neck is successively sutured directly with the urethra. The procedure is called urethrovesical anastomosis and is one of the most dexterity demanding tasks during RALP. Two suturing instruments and a pair of needles are used in combination to perform a running stitch during urethrovesical anastomosis. While robotic instruments provide enhanced dexterity to perform the anastomosis, it is still highly challenging and difficult to learn. In this paper, we presents a vision-guided needle grasping method for automatically grasping the needle that has been inserted into the patient prior to anastomosis. We aim to automatically grasp the suturing needle in a position that avoids hand-offs and immediately enables the start of suturing. The full grasping process can be broken down into: a needle detection algorithm; an approach phase where the surgical tool moves closer to the needle based on visual feedback; and a grasping phase through path planning based on observed surgical practice. Our experimental results show examples of successful autonomous grasping that has the potential to simplify and decrease the operational time in RALP by assisting a small component of urethrovesical anastomosis.

[1]  Fouad Aoun,et al.  The Single-Knot Running Vesicourethral Anastomosis after Minimally Invasive Prostatectomy: Review of the Technique and Its Modifications, Tips, and Pitfalls , 2016, Prostate cancer.

[2]  Peter Kazanzides,et al.  An open-source research kit for the da Vinci® Surgical System , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[3]  P. Hynes,et al.  Uncalibrated Visual-Servoing of a Dual-Arm Robot for MIS Suturing , 2006, The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006..

[4]  R. Hartley.  Triangulation, Computer Vision and Image Understanding , 1997 .

[5]  Ankush Gupta,et al.  A case study of trajectory transfer through non-rigid registration for a simplified suturing scenario , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  Ravishankar K. Iyer,et al.  Adverse Events in Robotic Surgery: A Retrospective Study of 14 Years of FDA Data , 2015, PloS one.

[7]  Peter I. Corke.  Integrating ROS and MATLAB [ROS Topics] , 2015, IEEE Robotics & Automation Magazine.

[8]  Kabir Yadav,et al.  The impact of wound age on the infection rate of simple lacerations repaired in the emergency department. , 2012, Injury.

[9]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[10]  S. Buss.  Introduction to Inverse Kinematics with Jacobian Transpose , Pseudoinverse and Damped Least Squares methods , 2004 .

[11]  Alois Knoll,et al.  Automation of tissue piercing using circular needles and vision guidance for computer aided laparoscopic surgery , 2010, 2010 IEEE International Conference on Robotics and Automation.

[12]  Alois Knoll,et al.  Autonomous High Precision Positioning of Surgical Instruments in Robot-Assisted Minimally Invasive Surgery under Visual Guidance , 2010, 2010 Sixth International Conference on Autonomic and Autonomous Systems.

[13]  Wyatt S. Newman,et al.  A novel vision guided knot-tying method for autonomous robotic surgery , 2014, 2014 IEEE International Conference on Automation Science and Engineering (CASE).

[14]  Gordon I. Dodds,et al.  Uncalibrated visual-servoing of a dual-arm robot for surgical tasks , 2005, 2005 International Symposium on Computational Intelligence in Robotics and Automation.

[15]  Matthew T. Gettman,et al.  Robotic Instrumentation, Personnel and Operating Room Setup , 2011 .

[16]  Russell H. Taylor,et al.  System for robot-assisted real-time laparoscopic ultrasound elastography , 2012, Medical Imaging.

[17]  Alexandru Tupan,et al.  Triangulation , 1997, Comput. Vis. Image Underst..

[18]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[19]  Pieter Abbeel,et al.  Autonomous multilateral debridement with the Raven surgical robot , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Kenneth Y. Goldberg,et al.  Automating multi-throw multilateral surgical suturing with a mechanical needle guide and sequential convex optimization , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[21]  K. M. Deliparaschos,et al.  Evolution of autonomous and semi‐autonomous robotic surgical systems: a review of the literature , 2011, The international journal of medical robotics + computer assisted surgery : MRCAS.

[22]  Brijen Thananjeyan,et al.  Multilateral surgical pattern cutting in 2D orthotropic gauze with deep reinforcement learning policies for tensioning , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Pieter Abbeel,et al.  Learning by observation for surgical subtasks: Multilateral cutting of 3D viscoelastic and 2D Orthotropic Tissue Phantoms , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[24]  John T. Wen,et al.  Autonomous suturing using minimally invasive surgical robots , 2000, Proceedings of the 2000. IEEE International Conference on Control Applications. Conference Proceedings (Cat. No.00CH37162).

[25]  Philippe Zanne,et al.  Visual Servoing-Based Endoscopic Path Following for Robot-Assisted Laparoscopic Surgery , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Bernard Espiau,et al.  Effect of Camera Calibration Errors on Visual Servoing in Robotics , 1993, ISER.

[27]  Gregory D. Hager,et al.  Human-Machine Collaborative surgery using learned models , 2011, 2011 IEEE International Conference on Robotics and Automation.

[28]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[29]  Ralph V Clayman,et al.  Technique for laparoscopic running urethrovesical anastomosis:the single knot method. , 2003, Urology.

[30]  Wyatt S. Newman,et al.  Automatic initialization and dynamic tracking of surgical suture threads , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[31]  Vibhav Vineet,et al.  Struck: Structured Output Tracking with Kernels , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Murat Cenk Cavusoglu,et al.  Modeling of needle-tissue interaction forces during surgical suturing , 2012, 2012 IEEE International Conference on Robotics and Automation.