EnViSoRS: Enhanced Vision System for Robotic Surgery. A User-Defined Safety Volume Tracking to Minimize the Risk of Intraoperative Bleeding

In abdominal surgery, intra-operative bleeding is one of the major complications that affect the outcome of minimally invasive surgical procedures. One of the causes is attributed to accidental damages to arteries or veins, and one of the possible risk factors falls on the surgeon's skills. This paper presents the development and application of an Enhanced Vision System for Robotic Surgery (EnViSoRS), based on a user-defined Safety Volume (SV) tracking to minimise the risk of intra-operative bleeding. It aims at enhancing the surgeon's capabilities by providing Augmented Reality (AR) assistance towards the protection of vessels from injury during the execution of surgical procedures with a robot. The core of the framework consists in: (i) a hybrid tracking algorithm (LT-SAT tracker) that robustly follows a user-defined Safety Area (SA) in long term; (ii) a dense soft tissue 3D reconstruction algorithm, necessary for the computation of the SV; (iii) AR features for visualisation of the SV to be protected and of a graphical gauge indicating the current distance between the instruments and the reconstructed surface. EnViSoRS was integrated with a commercial robotic surgery system (the dVRK system) for testing and validation. The experiments aimed at demonstrating the accuracy, robustness, performance and usability of EnViSoRS during the execution of a simulated surgical task on a liver phantom. Results show an overall accuracy in accordance with surgical requirements (< 5mm), and high robustness in the computation of the SV in terms of precision and recall of its identification. The optimisation strategy implemented to speed up the computational time is also described and evaluated, providing AR features update rate up to 4 fps without impacting the real-time visualisation of the stereo endoscopic video. Finally, qualitative results regarding the system usability indicate that the proposed system integrates well with the commercial surgical robot and has indeed potential to offer useful assistance during real surgeries.

[1]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[2]  Sebastian Bodenstedt,et al.  Dense GPU-enhanced surface reconstruction from stereo endoscopic images for intraoperative registration. , 2012, Medical physics.

[3]  Elena De Momi,et al.  Multi kinect people detection for intuitive and safe human robot cooperation in the operating room , 2013, 2013 16th International Conference on Advanced Robotics (ICAR).

[4]  Antonio M. Lacy,et al.  Robotics in General Surgery: Update and Future Perspectives , 2015, ICRA 2015.

[5]  Naoki Suzuki,et al.  Identification of inferior pancreaticoduodenal artery during pancreaticoduodenectomy using augmented reality‐based navigation system , 2014, Journal of hepato-biliary-pancreatic sciences.

[6]  Lena Maier-Hein,et al.  Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery , 2013, Medical Image Anal..

[7]  A. Lanfranco,et al.  Robotic Surgery: A Current Perspective , 2004, Annals of surgery.

[8]  Elena De Momi,et al.  Laparoscopic Tissue Retractor Based on Local Magnetic Actuation , 2015 .

[9]  Guang-Zhong Yang,et al.  Probabilistic Tracking of Affine-Invariant Anisotropic Regions , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Jiri Matas,et al.  Locally Optimized RANSAC , 2003, DAGM-Symposium.

[11]  Danail Stoyanov,et al.  Surgical Vision , 2011, Annals of Biomedical Engineering.

[12]  Darwin G. Caldwell,et al.  Comparative usability and performance evaluation of surgeon interfaces in laser phonomicrosurgery , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  J. Marescaux,et al.  Augmented reality in laparoscopic surgical oncology. , 2011, Surgical oncology.

[14]  Carl-Fredrik Westin,et al.  Speckle-constrained filtering of ultrasound images , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Ji Ma,et al.  A Modified Census Transform Based on the Neighborhood Information for Stereo Matching Algorithm , 2013, 2013 Seventh International Conference on Image and Graphics.

[16]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[17]  Russell H. Taylor,et al.  Fundus Image Mosaicking for Information Augmentation in Computer-Assisted Slit-Lamp Imaging , 2014, IEEE Transactions on Medical Imaging.

[18]  Michael C. Yip,et al.  Tissue Tracking and Registration for Image-Guided Surgery , 2012, IEEE Transactions on Medical Imaging.

[19]  Nima Enayati,et al.  Haptics in Robot-Assisted Surgery: Challenges and Benefits , 2016, IEEE Reviews in Biomedical Engineering.

[20]  Jeffrey A. Cadeddu,et al.  Toward Long-Term and Accurate Augmented-Reality for Monocular Endoscopic Videos , 2014, IEEE Transactions on Biomedical Engineering.

[21]  Guang-Zhong Yang,et al.  Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery , 2010, MICCAI.

[22]  Nikhil Deshpande,et al.  A novel computerized surgeon–machine interface for robot‐assisted laser phonomicrosurgery , 2014, The Laryngoscope.

[23]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[24]  Darwin G. Caldwell,et al.  Enhanced computer-assisted laser microsurgeries with a “virtual microscope” based surgical system , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[25]  C Freschi,et al.  Technical review of the da Vinci surgical telemanipulator , 2013, The international journal of medical robotics + computer assisted surgery : MRCAS.

[26]  Leo Joskowicz,et al.  Validation of a stereo camera system to quantify brain deformation due to breathing and pulsatility. , 2014, Medical physics.

[27]  Elena De Momi,et al.  Development of a surgical stereo endoscopic image dataset for validating 3D stereo reconstruction algorithms , 2016 .

[28]  Lena Maier-Hein,et al.  Comparative Validation of Single-Shot Optical Techniques for Laparoscopic 3-D Surface Reconstruction , 2014, IEEE Transactions on Medical Imaging.

[29]  Peter Kazanzides,et al.  An open-source research kit for the da Vinci® Surgical System , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[30]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[31]  Elena De Momi,et al.  Dense soft tissue 3D reconstruction refined with super-pixel segmentation for robotic abdominal surgery , 2016, International Journal of Computer Assisted Radiology and Surgery.

[32]  W. Gantert,et al.  Bleeding remains a major complication during laparoscopic surgery: analysis of the SALTS database , 2005, Langenbeck's Archives of Surgery.

[33]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[34]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.