Inter-foetus Membrane Segmentation for TTTS Using Adversarial Networks

Twin-to-Twin Transfusion Syndrome is commonly treated with minimally invasive laser surgery in fetoscopy. The inter-foetal membrane is used as a reference to find abnormal anastomoses. Membrane identification is a challenging task due to small field of view of the camera, presence of amniotic liquid, foetus movement, illumination changes and noise. This paper aims at providing automatic and fast membrane segmentation in fetoscopic images. We implemented an adversarial network consisting of two Fully-Convolutional Neural Networks. The former (the segmentor ) is a segmentation network inspired by U-Net and integrated with residual blocks, whereas the latter acts as critic and is made only of the encoding path of the segmentor . A dataset of 900 images acquired in 6 surgical cases was collected and labelled to validate the proposed approach. The adversarial networks achieved a median Dice similarity coefficient of 91.91% with Inter-Quartile Range ( IQR ) of 4.63%, overcoming approaches based on U-Net (82.98%- IQR : 14.41%) and U-Net with residual blocks (86.13%- IQR : 13.63%). Results proved that the proposed architecture could be a valuable and robust solution to assist surgeons in providing membrane identification while performing fetoscopic surgery.

[1]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[2]  K. Hecher,et al.  Laser coagulation of placental anastomoses with a 30° fetoscope in severe mid‐trimester twin–twin transfusion syndrome with anterior placenta , 2008, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[3]  R. Quintero,et al.  Twin-twin transfusion syndrome. , 2003, Clinics in perinatology.

[4]  Sébastien Ourselin,et al.  Towards computer-assisted TTTS: Laser ablation detection for workflow segmentation from fetoscopic video , 2018, International Journal of Computer Assisted Radiology and Surgery.

[5]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[6]  Vincent Dumoulin,et al.  Deconvolution and Checkerboard Artifacts , 2016 .

[7]  Jacob Chakareski,et al.  Effective Deep Learning for Semantic Segmentation Based Bleeding Zone Detection in Capsule Endoscopy Images , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[8]  Xinming Zhang,et al.  A novel framework for semantic segmentation with generative adversarial network , 2019, J. Vis. Commun. Image Represent..

[9]  Andrew L. Maas Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[10]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[11]  Sébastien Ourselin,et al.  Deep Sequential Mosaicking of Fetoscopic Videos , 2019, MICCAI.

[12]  Elena De Momi,et al.  Learning-based classification of informative laryngoscopic frames , 2018, Comput. Methods Programs Biomed..

[13]  Lena Maier-Hein,et al.  Uncertainty-Aware Organ Classification for Surgical Data Science Applications in Laparoscopy , 2017, IEEE Transactions on Biomedical Engineering.

[14]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Heung-Il Suk,et al.  Deep Learning in Medical Image Analysis. , 2017, Annual review of biomedical engineering.

[16]  Tao Xu,et al.  Adversarial learning with multi-scale loss for skin lesion segmentation , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[17]  Adrien Bartoli,et al.  Live image parsing in uterine laparoscopy , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[18]  Xenophon Papademetris,et al.  Deep-learned placental vessel segmentation for intraoperative video enhancement in fetoscopic surgery , 2018, International Journal of Computer Assisted Radiology and Surgery.

[19]  Hao Chen,et al.  Integrating Online and Offline Three-Dimensional Deep Learning for Automated Polyp Detection in Colonoscopy Videos , 2017, IEEE Journal of Biomedical and Health Informatics.

[20]  Derek Merck,et al.  Virtually Visualizing Vessels : A Study of the Annotation of Placental Vasculature from MRI in Large-scale Virtual Reality for Surgical Planning , 2016 .

[21]  Arianna Menciassi,et al.  Towards a Computed-Aided Diagnosis System in Colonoscopy: Automatic Polyp Segmentation Using Convolution Neural Networks , 2018, J. Medical Robotics Res..

[22]  Camille Couprie,et al.  Semantic Segmentation using Adversarial Networks , 2016, NIPS 2016.

[23]  Katrina Jane Williams,et al.  Twin-to-twin transfusion syndrome neurodevelopmental follow-up study (neurodevelopmental outcomes for children whose twin-to-twin transfusion syndrome was treated with placental laser photocoagulation) , 2018, BMC Pediatrics.

[24]  Danail Stoyanov,et al.  Articulated Multi-Instrument 2-D Pose Estimation Using Fully Convolutional Networks , 2018, IEEE Transactions on Medical Imaging.

[25]  W. Clewell,et al.  Twin-twin transfusion syndrome. , 1990, American journal of obstetrics and gynecology.

[26]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[27]  Chi Poissy,et al.  Endoscopic laser surgery versus serial amnioreduction for severe twin-to-twin transfusion syndrome* , 2004 .

[28]  Elli Angelopoulou,et al.  Automatic Region-of-Interest Segmentation and Pathology Detection in Magnetically Guided Capsule Endoscopy , 2011, MICCAI.

[29]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[30]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[31]  Sébastien Ourselin,et al.  A Combined EM and Visual Tracking Probabilistic Model for Robust Mosaicking: Application to Fetoscopy , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[32]  Michael Kampffmeyer,et al.  UNCERTAINTY MODELING AND INTERPRETABILITY IN CONVOLUTIONAL NEURAL NETWORKS FOR POLYP SEGMENTATION , 2018, 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP).

[33]  Syed Muhammad Anwar,et al.  Deep Learning in Medical Image Analysis , 2017 .

[34]  J. Deprest,et al.  Alternative technique for Nd : YAG laser coagulation in twin‐to‐twin transfusion syndrome with anterior placenta , 1998, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[35]  Pascal Getreuer,et al.  Automated vasculature extraction from placenta images , 2011, Medical Imaging.

[36]  J. Neilson,et al.  Interventions for the treatment of twin-twin transfusion syndrome. , 2014, The Cochrane database of systematic reviews.

[37]  Tao Xu,et al.  SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation , 2017, Neuroinformatics.

[38]  Pieter P. Jonker,et al.  Stable Image Registration for In-Vivo Fetoscopic Panorama Reconstruction , 2018, J. Imaging.

[39]  Danail Stoyanov,et al.  Deep Learning Based Robotic Tool Detection and Articulation Estimation With Spatio-Temporal Layers , 2019, IEEE Robotics and Automation Letters.

[40]  Adrien Bartoli,et al.  Automatic Detection of the Uterus and Fallopian Tube Junctions in Laparoscopic Images , 2015, IPMI.

[41]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[43]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[44]  S. Ourselin,et al.  Computer‐assisted surgical planning and intraoperative guidance in fetal surgery: a systematic review† , 2015, Prenatal diagnosis.

[45]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.