Autonomous Robotic Suction to Clear the Surgical Field for Hemostasis Using Image-Based Blood Flow Detection

Autonomous robotic surgery has seen significant progression over the last decade with the aims of reducing surgeon fatigue, improving procedural consistency, and perhaps one day take over surgery itself. However, automation has not been applied to the critical surgical task of controlling tissue and blood vessel bleeding--known as hemostasis. The task of hemostasis covers a spectrum of bleeding sources and a range of blood velocity, trajectory, and volume. In an extreme case, an un-controlled blood vessel fills the surgical field with flowing blood. In this work, we present the first, automated solution for hemostasis through development of a novel probabilistic blood flow detection algorithm and a trajectory generation technique that guides autonomous suction tools towards pooling blood. The blood flow detection algorithm is tested in both simulated scenes and in a real-life trauma scenario involving a hemorrhage that occurred during thyroidectomy. The complete solution is tested in a physical lab setting with the da Vinci Research Kit (dVRK) and a simulated surgical cavity for blood to flow through. The results show that our automated solution has accurate detection, a fast reaction time, and effective removal of the flowing blood. Therefore, the proposed methods are powerful tools to clearing the surgical field which can be followed by either a surgeon or future robotic automation developments to close the vessel rupture.

[1]  Elena De Momi,et al.  Automated Pick-Up of Suturing Needles for Robotic Surgical Assistance , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Christopher G. Atkeson,et al.  Stereo vision of liquid and particle flow for robot pouring , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[3]  Martial Hebert,et al.  Learning to Extract Motion from Videos in Convolutional Neural Networks , 2016, ACCV.

[4]  Guozheng Yan,et al.  Bleeding Detection in Wireless Capsule Endoscopy Based on Probabilistic Neural Network , 2011, Journal of Medical Systems.

[5]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[6]  Brijen Thananjeyan,et al.  Multilateral surgical pattern cutting in 2D orthotropic gauze with deep reinforcement learning policies for tensioning , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Lena Maier-Hein,et al.  2017 Robotic Instrument Segmentation Challenge , 2019, ArXiv.

[8]  Ryan K. Orosco,et al.  SuPer: A Surgical Perception Framework for Endoscopic Tissue Manipulation With Surgical Robotics , 2020, IEEE Robotics and Automation Letters.

[9]  A. Uhl,et al.  Computer-Aided Decision Support Systems for Endoscopy in the Gastrointestinal Tract: A Review , 2011, IEEE Reviews in Biomedical Engineering.

[10]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[11]  Yong-Sheng Chen,et al.  Pyramid Stereo Matching Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[12]  Michael C. Yip,et al.  Robot Autonomy for Surgery , 2017, The Encyclopedia of Medical Robotics.

[13]  Yun-Hui Liu,et al.  Dual-Arm Robotic Needle Insertion With Active Tissue Deformation for Autonomous Suturing , 2019, IEEE Robotics and Automation Letters.

[14]  Connor Schenck,et al.  Perceiving and reasoning about liquids using fully convolutional networks , 2017, Int. J. Robotics Res..

[15]  Wei Zhang,et al.  Computer-Aided Bleeding Detection in WCE Video , 2014, IEEE Journal of Biomedical and Health Informatics.

[16]  Takayuki Okamoto,et al.  Real-time identification of blood regions for hemostasis support in laparoscopic surgery , 2018, Signal Image Video Process..

[17]  Florian Richter,et al.  Open-Sourced Reinforcement Learning Environments for Surgical Robotics , 2019, ArXiv.

[18]  Connor Schenck,et al.  SPNets: Differentiable Fluid Dynamics for Deep Neural Networks , 2018, CoRL.

[19]  Pieter Abbeel,et al.  Learning by observation for surgical subtasks: Multilateral cutting of 3D viscoelastic and 2D Orthotropic Tissue Phantoms , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Gencheng Guo,et al.  Bleeding region detection in WCE images based on color features and neural network , 2011, 2011 IEEE 54th International Midwest Symposium on Circuits and Systems (MWSCAS).

[21]  Pieter Abbeel,et al.  Autonomous multilateral debridement with the Raven surgical robot , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[23]  Jong Hyo Kim,et al.  Active Blood Detection in a High Resolution Capsule Endoscopy using Color Spectrum Transformation , 2008, 2008 International Conference on BioMedical Engineering and Informatics.

[24]  Peter Kazanzides,et al.  An open-source research kit for the da Vinci® Surgical System , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Jianguo Liu,et al.  Obscure bleeding detection in endoscopy images using support vector machines , 2009 .

[26]  Ali Farhadi,et al.  See the Glass Half Full: Reasoning About Liquid Containers, Their Volume and Content , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[27]  Kenneth Y. Goldberg,et al.  Automating multi-throw multilateral surgical suturing with a mechanical needle guide and sequential convex optimization , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[28]  Max Q.-H. Meng,et al.  Computer-Aided Detection of Bleeding Regions for Capsule Endoscopy Images , 2009, IEEE Transactions on Biomedical Engineering.

[29]  Florian Richter,et al.  SuPer Deep: A Surgical Perception Framework for Robotic Tissue Manipulation using Deep Learning for Feature Extraction , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).