NephCNN: A deep-learning framework for vessel segmentation in nephrectomy laparoscopic videos

Objective: In the last years, Robot-assisted partial nephrectomy (RAPN) is establishing as elected treatment for renal cell carcinoma (RCC). Reduced field of view, field occlusions by surgical tools, and reduced maneuverability may potentially cause accidents, such as unwanted vessel resection with consequent bleeding. Surgical Data Science (SDS) can provide effective context-aware tools for supporting surgeons. However, currently no tools have been exploited for automatic vessels segmentation from nephrectomy laparoscopic videos. Herein, we propose a new approach based on adversarial Fully Convolutional Neural Networks (FCNNs) to kidney vessel segmentation from nephrectomy laparoscopic vision. Methods: The proposed approach enhances existing segmentation framework by (i) encoding 3D kernels for spatio-temporal features extraction to enforce pixel connectivity in time, and (ii) perform training in adversarial fashion, which constrains vessels shape. Results: We performed a preliminary study using 8 different RAPN videos (1871 frames), the first in the field, achieving a median Dice Similarity Coefficient of 71.76%. Conclusions: Results showed that the proposed approach could be a valuable solution with a view to assist surgeon during RAPN.

[1]  Elena De Momi,et al.  Long Term Safety Area Tracking (LT‐SAT) with online failure detection and recovery for robotic minimally invasive surgery , 2018, Medical Image Anal..

[2]  Keno März,et al.  Toward a standard ontology of surgical process models , 2018, International Journal of Computer Assisted Radiology and Surgery.

[3]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[4]  Emanuele Frontoni,et al.  Inter-foetus Membrane Segmentation for TTTS Using Adversarial Networks , 2019, Annals of Biomedical Engineering.

[5]  P. Tamboli,et al.  Identifying the risk of disease progression after surgery for localized renal cell carcinoma , 2010, BJU international.

[6]  A. Gavin,et al.  Cancer incidence and mortality patterns in Europe: Estimates for 40 countries and 25 major cancers in 2018. , 2018, European journal of cancer.

[7]  Gongping Yang,et al.  Hierarchical retinal blood vessel segmentation based on feature and ensemble learning , 2015, Neurocomputing.

[8]  Cameron N. Riviere,et al.  Toward Improving Safety in Neurosurgery with an Active Handheld Instrument , 2018, Annals of Biomedical Engineering.

[9]  N’Dowa.  Systematic review of oncological outcomes following surgical management of localised renal cancer , 2012 .

[10]  P. Dasgupta,et al.  ROBOT‐ASSISTED PARTIAL NEPHRECTOMY , 2008, BJU international.

[11]  Tianfu Wang,et al.  A Cross-Modality Learning Approach for Vessel Segmentation in Retinal Images , 2016, IEEE Transactions on Medical Imaging.

[12]  J. Ferlay,et al.  Cancer incidence and mortality patterns in Europe: estimates for 40 countries in 2012. , 2013, European journal of cancer.

[13]  Pabitra Mitra,et al.  Generative Adversarial Learning for Reducing Manual Annotation in Semantic Segmentation on Large Scale Miscroscopy Images: Automated Vessel Segmentation in Retinal Fundus Image as Test Case , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[15]  Lena Maier-Hein,et al.  Uncertainty-Aware Organ Classification for Surgical Data Science Applications in Laparoscopy , 2017, IEEE Transactions on Biomedical Engineering.

[16]  Lei Zhang,et al.  Multi-level deep supervised networks for retinal vessel segmentation , 2017, International Journal of Computer Assisted Radiology and Surgery.

[17]  Yuri Sousa Aurelio,et al.  Learning from Imbalanced Data Sets with Weighted Cross-Entropy Function , 2019, Neural Processing Letters.

[18]  Victor S. Lempitsky,et al.  N4-Fields: Neural Network Nearest Neighbor Fields for Image Transforms , 2014, ArXiv.

[19]  David J. Kriegman,et al.  Dense Volume-to-Volume Vascular Boundary Detection , 2016, MICCAI.

[20]  Sonam Singh,et al.  A fully convolutional neural network based structured prediction approach towards the retinal vessel segmentation , 2016, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[21]  Sang Jun Park,et al.  Retinal Vessel Segmentation in Fundoscopic Images with Generative Adversarial Networks , 2017, ArXiv.

[22]  Sara Moccia,et al.  EndoAbS dataset: Endoscopic abdominal stereo image dataset for benchmarking 3D stereo reconstruction algorithms , 2018, The international journal of medical robotics + computer assisted surgery : MRCAS.

[23]  Elena De Momi,et al.  Blood vessel segmentation algorithms - Review of methods, datasets and evaluation metrics , 2018, Comput. Methods Programs Biomed..

[24]  Danail Stoyanov,et al.  Deep Learning Based Robotic Tool Detection and Articulation Estimation With Spatio-Temporal Layers , 2019, IEEE Robotics and Automation Letters.