Learning normalized inputs for iterative estimation in medical image segmentation

HighlightsImage segmentation pipeline based on Fully Convolutional Networks (FCN) and ResNets is proposed.FCN can serve as a pre‐processor to normalize medical imaging input data.A trainable FCN is an alternative to hand‐designed, modality specific pre‐processing steps.Our pipeline obtains or matches state‐of‐the‐art performance on 3 segmentation datasets. Graphical abstract Figure. No Caption available. Abstract In this paper, we introduce a simple, yet powerful pipeline for medical image segmentation that combines Fully Convolutional Networks (FCNs) with Fully Convolutional Residual Networks (FC‐ResNets). We propose and examine a design that takes particular advantage of recent advances in the understanding of both Convolutional Neural Networks as well as ResNets. Our approach focuses upon the importance of a trainable pre‐processing when using FC‐ResNets and we show that a low‐capacity FCN model can serve as a pre‐processor to normalize medical input data. In our image segmentation pipeline, we use FCNs to obtain normalized images, which are then iteratively refined by means of a FC‐ResNet to generate a segmentation prediction. As in other fully convolutional approaches, our pipeline can be used off‐the‐shelf on different image modalities. We show that using this pipeline, we exhibit state‐of‐the‐art performance on the challenging Electron Microscopy benchmark, when compared to other 2D methods. We improve segmentation results on CT images of liver lesions, when contrasting with standard FCN methods. Moreover, when applying our 2D pipeline on a challenging 3D MRI prostate segmentation challenge we reach results that are competitive even when compared to 3D methods. The obtained results illustrate the strong potential and versatility of the pipeline by achieving accurate segmentations on a variety of image modalities and different anatomical regions.

[1]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Ronald M. Summers,et al.  DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation , 2015, MICCAI.

[3]  Jürgen Schmidhuber,et al.  Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation , 2015, NIPS.

[4]  P. Cattin,et al.  Multi-dimensional Gated Recurrent Units for the Segmentation of Biomedical 3D-Data , 2016, LABELS/DLMIA@MICCAI.

[5]  Hao Chen,et al.  Deep Contextual Networks for Neuronal Structure Segmentation , 2016, AAAI.

[6]  Xundong Wu.  An Iterative Convolutional Neural Network Algorithm Improves Electron Microscopy Image Segmentation , 2015, ArXiv.

[7]  Mohammad Havaei,et al.  Deep Learning Trends for Focal Brain Pathology Segmentation in MRI , 2016, Machine Learning for Health Informatics.

[8]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[9]  Jialin Peng,et al.  Automatic 3D liver segmentation based on deep learning and globally optimized surface evolution , 2016, Physics in medicine and biology.

[10]  S. Dwivedi,et al.  Obesity May Be Bad: Compressed Convolutional Networks for Biomedical Image Segmentation , 2020 .

[11]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[12]  Chengwen Chu,et al.  Fully Automatic Localization and Segmentation of 3D Vertebral Bodies from CT/MR Images via a Learning-Based Method , 2015, PloS one.

[13]  Hao Chen,et al.  3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes , 2016, MICCAI.

[14]  Serge J. Belongie,et al.  Residual Networks Behave Like Ensembles of Relatively Shallow Networks , 2016, NIPS.

[15]  Fred A. Hamprecht,et al.  Multi-modal Brain Tumor Segmentation using Deep Convolutional Neural Networks , 2014 .

[16]  Shuiwang Ji,et al.  Residual Deconvolutional Networks for Brain Electron Microscopy Image Segmentation , 2017, IEEE Transactions on Medical Imaging.

[17]  Joachim M. Buhmann,et al.  Crowdsourcing the creation of image segmentation algorithms for connectomics , 2015, Front. Neuroanat..

[18]  Seyed-Ahmad Ahmadi,et al.  Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields , 2016, MICCAI.

[19]  Hao Chen,et al.  VoxResNet: Deep Voxelwise Residual Networks for Volumetric Brain Segmentation , 2016, ArXiv.

[20]  Chao Chen,et al.  Optree: A Learning-Based Adaptive Watershed Algorithm for Neuron Segmentation , 2014, MICCAI.

[21]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[23]  Leo Joskowicz,et al.  Automatic Liver Tumor Segmentation in Follow-Up CT Scans: Preliminary Method and Results , 2015, Patch-MI@MICCAI.

[24]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Ullrich Köthe,et al.  An Efficient Fusion Move Algorithm for the Minimum Cost Lifted Multicut Problem , 2016, ECCV.

[26]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[27]  Kilian Q. Weinberger,et al.  Deep Networks with Stochastic Depth , 2016, ECCV.

[28]  Daguang Xu,et al.  Deep Learning Based Automatic Segmentation of Pathological Kidney in CT: Local Versus Global Image Context , 2017, Deep Learning and Convolutional Neural Networks for Medical Image Computing.

[29]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[30]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[31]  Tomaso A. Poggio,et al.  Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex , 2016, ArXiv.

[32]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[33]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[34]  Hayit Greenspan,et al.  Fully Convolutional Network for Liver Segmentation and Lesions Detection , 2016, LABELS/DLMIA@MICCAI.

[35]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[36]  Ronald M. Summers,et al.  Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation a set of bounding boxes covering each image superpixel at multiple spatial scales in a “ zoom-out ” fashion , 2018 .

[37]  John Salvatier,et al.  Theano: A Python framework for fast computation of mathematical expressions , 2016, ArXiv.

[38]  Fucang Jia,et al.  Automatic Segmentation of Liver Tumor in CT Images with Deep Convolutional Neural Networks , 2015 .

[39]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[40]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[41]  Lubomir M. Hadjiiski,et al.  Urinary bladder segmentation in CT urography using deep-learning convolutional neural network and level sets. , 2016, Medical physics.

[42]  Won-Ki Jeong,et al.  FusionNet: A Deep Fully Residual Convolutional Neural Network for Image Segmentation in Connectomics , 2016, Frontiers in Computer Science.

[43]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[44]  Ting Liu,et al.  A modular hierarchical approach to 3D electron microscopy image segmentation , 2014, Journal of Neuroscience Methods.

[45]  Tom Gundersen,et al.  Nabla-net: A Deep Dag-Like Convolutional Architecture for Biomedical Image Segmentation , 2016, BrainLes@MICCAI.

[46]  Yong Fan,et al.  Brain Tumor Segmentation Using a Fully Convolutional Neural Network with Conditional Random Fields , 2016, BrainLes@MICCAI.

[47]  Jürgen Schmidhuber,et al.  Highway and Residual Networks learn Unrolled Iterative Estimation , 2016, ICLR.

[48]  B. Ginneken,et al.  3D Segmentation in the Clinic: A Grand Challenge , 2007 .

[49]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[50]  Stephan Miller,et al.  A system for rapid prototyping of hearts with congenital malformations based on the medical imaging interaction toolkit (MITK) , 2006, SPIE Medical Imaging.

[51]  Hayit Greenspan,et al.  Longitudinal Multiple Sclerosis Lesion Segmentation Using Multi-view Convolutional Neural Networks , 2016, LABELS/DLMIA@MICCAI.

[52]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[53]  Bostjan Likar,et al.  Model-Based Segmentation of Vertebral Bodies from MR Images with 3D CNNs , 2016, MICCAI.

[54]  Hao Chen,et al.  Volumetric ConvNets with Mixed Residual Connections for Automated Prostate Segmentation from 3D MR Images , 2017, AAAI.

[55]  Serge J. Belongie,et al.  Residual Networks are Exponential Ensembles of Relatively Shallow Networks , 2016, ArXiv.

[56]  Christopher Joseph Pal,et al.  Convolutional networks for kidney segmentation in contrast-enhanced CT scans , 2018, Comput. methods Biomech. Biomed. Eng. Imaging Vis..