Toward Improving Safety in Neurosurgery with an Active Handheld Instrument

Microsurgical procedures, such as petroclival meningioma resection, require careful surgical actions in order to remove tumor tissue, while avoiding brain and vessel damaging. Such procedures are currently performed under microscope magnification. Robotic tools are emerging in order to filter surgeons’ unintended movements and prevent tools from entering forbidden regions such as vascular structures. The present work investigates the use of a handheld robotic tool (Micron) to automate vessel avoidance in microsurgery. In particular, we focused on vessel segmentation, implementing a deep-learning-based segmentation strategy in microscopy images, and its integration with a feature-based passive 3D reconstruction algorithm to obtain accurate and robust vessel position. We then implemented a virtual-fixture-based strategy to control the handheld robotic tool and perform vessel avoidance. Clay vascular phantoms, lying on a background obtained from microscopy images recorded during petroclival meningioma surgery, were used for testing the segmentation and control algorithms. When testing the segmentation algorithm on 100 different phantom images, a median Dice similarity coefficient equal to 0.96 was achieved. A set of 25 Micron trials of 80 s in duration, each involving the interaction of Micron with a different vascular phantom, were recorded, with a safety distance equal to 2 mm, which was comparable to the median vessel diameter. Micron’s tip entered the forbidden region 24% of the time when the control algorithm was active. However, the median penetration depth was 16.9 μm, which was two orders of magnitude lower than median vessel diameter. Results suggest the system can assist surgeons in performing safe vessel avoidance during neurosurgical procedures.

[1]  Tae Hyun Kim,et al.  Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Jess H Lonner,et al.  Robotically Assisted Unicompartmental Knee Arthroplasty with a Handheld Image-Free Sculpting Tool. , 2016, The Orthopedic clinics of North America.

[3]  Sven Loncaric,et al.  Segmentation of the foveal microvasculature using deep learning networks , 2016, Journal of biomedical optics.

[4]  Gregory S. Corrado,et al.  Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning , 2017, Nature Biomedical Engineering.

[5]  Daniel P. Huttenlocher,et al.  Distance Transforms of Sampled Functions , 2012, Theory Comput..

[6]  Elena De Momi,et al.  Learning-based classification of informative laryngoscopic frames , 2018, Comput. Methods Programs Biomed..

[7]  Elena De Momi,et al.  Blood vessel segmentation algorithms - Review of methods, datasets and evaluation metrics , 2018, Comput. Methods Programs Biomed..

[8]  Michael T Lawton,et al.  Incidence and Predictors of Complications and Mortality in Cerebrovascular Surgery: National Trends From 2007 to 2012. , 2016, Neurosurgery.

[9]  Ching Y. Suen,et al.  Thinning Methodologies - A Comprehensive Survey , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Krzysztof Krawiec,et al.  Segmenting Retinal Blood Vessels With Deep Neural Networks , 2016, IEEE Transactions on Medical Imaging.

[11]  Russell H. Taylor,et al.  Preliminary evaluation of a micro-force sensing handheld robot for vitreoretinal surgery , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Garnette R. Sutherland,et al.  Why Robots Entered Neurosurgery , 2016 .

[13]  Ketan R Bulsara,et al.  Surgery for petroclival meningiomas: a comprehensive review of outcomes in the skull base surgery era. , 2010, Skull base : official journal of North American Skull Base Society ... [et al.].

[14]  Giancarlo Ferrigno,et al.  Nonlinear Force Feedback Enhancement for Cooperative Robotic Neurosurgery Enforces Virtual Boundaries on Cortex Surface , 2016, J. Medical Robotics Res..

[15]  Lei Zhang,et al.  Multi-level deep supervised networks for retinal vessel segmentation , 2017, International Journal of Computer Assisted Radiology and Surgery.

[16]  Giancarlo Ferrigno,et al.  Adaptive Hands-On Control for Reaching and Targeting Tasks in Surgery , 2015 .

[17]  Alejandro F. Frangi,et al.  Muliscale Vessel Enhancement Filtering , 1998, MICCAI.

[18]  Sanju Lama,et al.  The evolution of neuroArm. , 2013, Neurosurgery.

[19]  Andreas Hackethal,et al.  Handheld articulating laparoscopic instruments driven by robotic technology. First clinical experience in gynecological surgery , 2012, Gynecological Surgery.

[20]  Zhenan Sun,et al.  Accurate iris segmentation in non-cooperative environments using fully convolutional networks , 2016, 2016 International Conference on Biometrics (ICB).

[21]  Yue-Fang Chang,et al.  An investigation into quality of life improvement in patients undergoing microvascular decompression for hemifacial spasm. , 2018, Journal of neurosurgery.

[22]  Dominiek Reynaerts,et al.  Development and experimental validation of a force sensing needle for robotically assisted retinal vein cannulations , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Cameron N. Riviere,et al.  Toward automated intraocular laser surgery using a handheld micromanipulator , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Sonam Singh,et al.  A fully convolutional neural network based structured prediction approach towards the retinal vessel segmentation , 2016, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[25]  Lei Zhang,et al.  Retinal vessel extraction by matched filter with first-order derivative of Gaussian , 2010, Comput. Biol. Medicine.

[26]  Berk Gonenc,et al.  Intraocular snake integrated with the steady-hand eye robot for assisted retinal microsurgery , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[27]  Hugh F. Durrant-Whyte,et al.  Simultaneous localization and mapping: part I , 2006, IEEE Robotics & Automation Magazine.

[28]  Guang-Zhong Yang,et al.  Dimensionality Reduction in Controlling Articulated Snake Robot for Endoscopy Under Dynamic Active Constraints , 2013, IEEE Transactions on Robotics.

[29]  Louis A. Lobes,et al.  Vision-Based Control of a Handheld Surgical Micromanipulator With Virtual Fixtures , 2013, IEEE Transactions on Robotics.

[30]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Gongping Yang,et al.  Hierarchical retinal blood vessel segmentation based on feature and ensemble learning , 2015, Neurocomputing.

[32]  Mark Hallett,et al.  Microvascular decompression for hemifacial spasm in patients >65 years of age: An analysis of outcomes and complications , 2013, Muscle & nerve.

[33]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[34]  Bunyarit Uyyanonvara,et al.  Blood vessel segmentation methodologies in retinal images - A survey , 2012, Comput. Methods Programs Biomed..

[35]  David J. Kriegman,et al.  Dense Volume-to-Volume Vascular Boundary Detection , 2016, MICCAI.

[36]  Garnette R. Sutherland,et al.  NeuroArm: an MR compatible robot for microsurgery , 2003, CARS.

[37]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[38]  Yue Chen,et al.  GPU-based proximity query processing on unstructured triangular mesh model , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[39]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[40]  David G. Lowe,et al.  Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration , 2009, VISAPP.

[41]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[42]  Erik Smistad,et al.  Vessel Detection in Ultrasound Images Using Deep Convolutional Neural Networks , 2016, LABELS/DLMIA@MICCAI.

[43]  Hui Feng,et al.  CNN-SVM for Microvascular Morphological Type Recognition with Data Augmentation , 2016, Journal of Medical and Biological Engineering.

[44]  Michael J. Black,et al.  Occlusion Boundary Detection via Deep Exploration of Context , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  M. Mitsuishi,et al.  Microsurgical robotic system for the deep surgical field: development of a prototype and feasibility studies in animal and cadaveric models. , 2005, Journal of neurosurgery.

[46]  R. R. Smith,et al.  Petrosal approach for petroclival meningiomas. , 1988, Neurosurgery.

[47]  Tianfu Wang,et al.  A Cross-Modality Learning Approach for Vessel Segmentation in Retinal Images , 2016, IEEE Transactions on Medical Imaging.

[48]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[49]  Johannes Lang,et al.  Clinical Anatomy of the Head: Neurocranium · Orbit · Craniocervical Regions , 1983 .

[50]  Michael V. McConnell,et al.  Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning , 2017, Nature Biomedical Engineering.

[51]  Victor S. Lempitsky,et al.  N4-Fields: Neural Network Nearest Neighbor Fields for Image Transforms , 2014, ArXiv.

[52]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[53]  Andru Putra Twinanda,et al.  EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos , 2016, IEEE Transactions on Medical Imaging.

[54]  Luc Van Gool,et al.  Deep Retinal Image Understanding , 2016, MICCAI.

[55]  Cameron N. Riviere,et al.  Manipulator Design and Operation of a Six-Degree-of-Freedom Handheld Tremor-Canceling Microsurgical Instrument , 2015, IEEE/ASME Transactions on Mechatronics.

[56]  Stephen Lin,et al.  DeepVessel: Retinal Vessel Segmentation via Deep Learning and Conditional Random Field , 2016, MICCAI.

[57]  Yang Liu,et al.  Vessel wall differences between middle cerebral artery and basilar artery plaques on magnetic resonance imaging , 2016, Scientific Reports.

[58]  Yadong Wang,et al.  Accurate Vessel Segmentation With Constrained B-Snake , 2015, IEEE Transactions on Image Processing.

[59]  M. Goldbaum,et al.  Detection of blood vessels in retinal images using two-dimensional matched filters. , 1989, IEEE transactions on medical imaging.

[60]  Cameron N. Riviere,et al.  Design and analysis of 6 DOF handheld micromanipulator , 2012, 2012 IEEE International Conference on Robotics and Automation.

[61]  Sameh A. Salem,et al.  Segmentation of retinal blood vessels based on analysis of the hessian matrix and Clustering Algorithm , 2007, 2007 15th European Signal Processing Conference.

[62]  Cameron N. Riviere,et al.  High-Speed Microscale Optical Tracking Using Digital Frequency-Domain Multiplexing , 2009, IEEE Transactions on Instrumentation and Measurement.

[63]  Kurt Novak,et al.  Precision rectification of SPOT imagery using the direct linear transformation model , 1996 .

[64]  Daniel Braun,et al.  EyeSLAM: Real‐time simultaneous localization and mapping of retinal vessels during intraocular microsurgery , 2018, The international journal of medical robotics + computer assisted surgery : MRCAS.

[65]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[66]  Cameron N. Riviere,et al.  Micron: An Actively Stabilized Handheld Tool for Microsurgery , 2012, IEEE Transactions on Robotics.

[67]  Dinesh Manocha,et al.  Collision-free and smooth trajectory computation in cluttered environments , 2012, Int. J. Robotics Res..