Vision-based Virtual Fixtures Generation for Robotic-Assisted Polyp Dissection Procedures

Polyp dissection requires very accurate detection of the region of interest and high-precision cutting with adequate safety margins. Robot-assisted polyp dissection is a solution to accomplish high-quality intervention. This paper proposes a method to constrain the robot to follow an accurate dissection path based on Virtual Fixtures (VF). The VFs are created via specific control points obtained directly from images of the surgical scene and are updated by the vision algorithm. The VF constraints can autonomously adapt themselves to environment changing during the surgical intervention. The entire pipeline is validated through experiments on the da Vinci Research Kit (dVRK) robot.

[1]  Peter Kazanzides,et al.  An open-source research kit for the da Vinci® Surgical System , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Alberto Bartoli,et al.  Robot-assisted laparoscopic total and partial gastric resection with D2 lymph node dissection for adenocarcinoma , 2008, Surgical Endoscopy.

[3]  Louis B. Rosenberg,et al.  Virtual fixtures: Perceptual tools for telerobotic manipulation , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[4]  B. Siciliano,et al.  Vision-based Virtual Fixtures Generation for MIRS Dissection Tasks , 2019 .

[5]  Vincenzo Lippiello,et al.  Real-time tracking of 3D elastic objects with an RGB-D sensor , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Bruno Siciliano,et al.  Passive Virtual Fixtures Adaptation in Minimally Invasive Robotic Surgery , 2018, IEEE Robotics and Automation Letters.

[7]  Alessandro De Luca,et al.  Sensorless Robot Collision Detection and Hybrid Force/Motion Control , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[8]  Zhengrong Liang,et al.  Improved Curvature Estimation for Shape Analysis in Computer-Aided Detection of Colonic Polyps , 2010, Virtual Colonoscopy and Abdominal Imaging.

[9]  Dimitris A. Karras,et al.  Computer-aided tumor detection in endoscopic video using color wavelet features , 2003, IEEE Transactions on Information Technology in Biomedicine.

[10]  Andreas Uhl,et al.  Colonic Polyp Classification with Convolutional Neural Networks , 2016, 2016 IEEE 29th International Symposium on Computer-Based Medical Systems (CBMS).

[11]  Bruno Siciliano,et al.  Modelling and identification of the da Vinci Research Kit robotic arms , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[12]  Paul Fockens,et al.  Colorectal polypectomy and endoscopic mucosal resection (EMR): European Society of Gastrointestinal Endoscopy (ESGE) Clinical Guideline , 2017, Endoscopy.

[13]  Howard Jay Chizeck,et al.  Forbidden-region virtual fixtures from streaming point clouds: Remotely touching and protecting a beating heart , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Sae Hwang,et al.  Polyp detection in Wireless Capsule Endoscopy videos based on image segmentation and geometric feature , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[15]  G. Oriolo,et al.  Robotics: Modelling, Planning and Control , 2008 .

[16]  Alessandro De Luca,et al.  Actuator failure detection and isolation using generalized momenta , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[17]  Gregory D. Hager,et al.  Vision-assisted control for manipulation using virtual fixtures , 2001, IEEE Transactions on Robotics.

[18]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[19]  David G. Lowe,et al.  Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration , 2009, VISAPP.

[20]  Danail Stoyanov,et al.  Surgical Vision , 2011, Annals of Biomedical Engineering.

[21]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[22]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[23]  Sae Hwang.  Bag-of-Visual-Words Approach to Abnormal Image Detection in Wireless Capsule Endoscopy Videos , 2011, ISVC.

[24]  Brian L. Davies,et al.  Active Constraints/Virtual Fixtures: A Survey , 2014, IEEE Transactions on Robotics.

[25]  Nima Enayati,et al.  Haptics in Robot-Assisted Surgery: Challenges and Benefits , 2016, IEEE Reviews in Biomedical Engineering.

[26]  Timothy N Judkins,et al.  Augmented reality and haptic interfaces for robot‐assisted surgery , 2012, The international journal of medical robotics + computer assisted surgery : MRCAS.

[27]  Aymeric Histace,et al.  Comparative Validation of Polyp Detection Methods in Video Colonoscopy: Results From the MICCAI 2015 Endoscopic Vision Challenge , 2017, IEEE Transactions on Medical Imaging.