CE-Net: Context Encoder Network for 2D Medical Image Segmentation

Medical image segmentation is an important step in medical image analysis. With the rapid development of a convolutional neural network in image processing, deep learning has been used for medical image segmentation, such as optic disc segmentation, blood vessel detection, lung segmentation, cell segmentation, and so on. Previously, U-net based approaches have been proposed. However, the consecutive pooling and strided convolutional operations led to the loss of some spatial information. In this paper, we propose a context encoder network (CE-Net) to capture more high-level information and preserve spatial information for 2D medical image segmentation. CE-Net mainly contains three major components: a feature encoder module, a context extractor, and a feature decoder module. We use the pretrained ResNet block as the fixed feature extractor. The context extractor module is formed by a newly proposed dense atrous convolution block and a residual multi-kernel pooling block. We applied the proposed CE-Net to different 2D medical image segmentation tasks. Comprehensive results show that the proposed method outperforms the original U-Net method and other state-of-the-art methods for optic disc segmentation, vessel detection, lung segmentation, cell contour segmentation, and retinal optical coherence tomography layer segmentation.

[1]  Raphael Sznitman,et al.  Pathological OCT Retinal Layer Segmentation Using Branch Residual U-Shape Networks , 2017, MICCAI.

[2]  Francisco Fumero,et al.  RIM-ONE: An open retinal image database for optic nerve evaluation , 2011, 2011 24th International Symposium on Computer-Based Medical Systems (CBMS).

[3]  Hans-Peter Meinzer,et al.  A Statistical Deformable Model for the Segmentation of Liver CT Volumes , 2007 .

[4]  Yousef Al-Kofahi,et al.  Improved Automatic Detection and Segmentation of Cell Nuclei in Histopathology Images , 2010, IEEE Transactions on Biomedical Engineering.

[5]  Zhenyu Liu,et al.  Central focused convolutional neural networks: Developing a data-driven model for lung nodule segmentation , 2017, Medical Image Anal..

[6]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[7]  Nassir Navab,et al.  ReLayNet: Retinal Layer and Fluid Segmentation of Macular Optical Coherence Tomography using Fully Convolutional Network , 2017, Biomedical optics express.

[8]  Shenghua Gao,et al.  Multi-Cell Multi-Task Convolutional Neural Networks for Diabetic Retinopathy Grading , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[9]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Yen-Wei Chen,et al.  Liver segmentation using superpixel-based graph cuts and restricted regions of shape constrains , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[11]  Xiaochun Cao,et al.  Joint Optic Disc and Cup Segmentation Based on Multi-Label Deep Network and Polar Transformation , 2018, IEEE Transactions on Medical Imaging.

[12]  A. Sevastopolsky,et al.  Optic disc and cup segmentation methods for glaucoma detection with modification of U-Net convolutional neural network , 2017, Pattern Recognition and Image Analysis.

[13]  Krzysztof Krawiec,et al.  Segmenting Retinal Blood Vessels With Deep Neural Networks , 2016, IEEE Transactions on Medical Imaging.

[14]  Oscar Camara,et al.  Generalized Overlap Measures for Evaluation and Validation in Medical Image Analysis , 2006, IEEE Transactions on Medical Imaging.

[15]  Keshab K. Parhi,et al.  Iterative Vessel Segmentation of Fundus Images , 2015, IEEE Transactions on Biomedical Engineering.

[16]  Zhenfeng Zhang,et al.  Superpixel-Based Segmentation for 3D Prostate MR Images , 2016, IEEE Transactions on Medical Imaging.

[17]  W. Eric L. Grimson,et al.  A shape-based approach to the segmentation of medical imagery using level sets , 2003, IEEE Transactions on Medical Imaging.

[18]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[19]  Xiangyu Zhang,et al.  Large Kernel Matters — Improve Semantic Segmentation by Global Convolutional Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[22]  Shuicheng Yan,et al.  Learning supervised descent directions for optic disc segmentation , 2018, Neurocomputing.

[23]  Tien Yin Wong,et al.  Superpixel Classification Based Optic Disc and Optic Cup Segmentation for Glaucoma Screening , 2013, IEEE Transactions on Medical Imaging.

[24]  Daniel Rueckert,et al.  Discriminative dictionary learning for abdominal multi-organ segmentation , 2015, Medical Image Anal..

[25]  Dimitris N. Metaxas,et al.  Image Segmentation Based on the Integration of Markov Random Fields and Deformable Models , 2000, MICCAI.

[26]  Rangaraj M. Rangayyan,et al.  Detection of the optic disc in images of the retina using the Hough transform , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[27]  Nassir Navab,et al.  InfiNet: Fully convolutional networks for infant brain MRI segmentation , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[28]  Xiaogang Wang,et al.  Zoom-in-Net: Deep Mining Lesions for Diabetic Retinopathy Detection , 2017, MICCAI.

[29]  Venkateswararao Cherukuri,et al.  Learning Based Segmentation of CT Brain Images: Application to Postoperative Hydrocephalic Scans , 2017, IEEE Transactions on Biomedical Engineering.

[30]  A. Cardona,et al.  An Integrated Micro- and Macroarchitectural Analysis of the Drosophila Brain by Computer-Assisted Serial Section Electron Microscopy , 2010, PLoS biology.

[31]  Dacheng Tao,et al.  Speckle Reduction in 3D Optical Coherence Tomography of Retina by A-Scan Reconstruction , 2016, IEEE Trans. Medical Imaging.

[32]  Rebecca Smith,et al.  Automated ventricular systems segmentation in brain CT images by combining low-level segmentation and high-level template matching , 2009, BMC Medical Informatics Decis. Mak..

[33]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Nathan Srebro,et al.  The Marginal Value of Adaptive Gradient Methods in Machine Learning , 2017, NIPS.

[36]  Young H. Kwon,et al.  Automated segmentation of the optic disc from stereo color photographs using physiologically plausible features. , 2007, Investigative ophthalmology & visual science.

[37]  Juan Xu,et al.  Optic disk feature extraction via modified deformable model technique for glaucoma analysis , 2007, Pattern Recognit..

[38]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[39]  Hao Chen,et al.  VoxResNet: Deep Voxelwise Residual Networks for Volumetric Brain Segmentation , 2016, ArXiv.

[40]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[41]  Georgy L. Gimel'farb,et al.  A new deformable model-based segmentation approach for accurate extraction of the kidney from abdominal CT images , 2011, 2011 18th IEEE International Conference on Image Processing.

[42]  Andrew Hunter,et al.  Optic nerve head segmentation , 2004, IEEE Transactions on Medical Imaging.

[43]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[44]  Jun Cheng,et al.  Sparse Range-Constrained Learning and Its Application for Medical Image Grading , 2018, IEEE Transactions on Medical Imaging.

[45]  Daniel Rueckert,et al.  Automatic Whole Brain MRI Segmentation of the Developing Neonatal Brain , 2014, IEEE Transactions on Medical Imaging.

[46]  S. Majumdar,et al.  Use of 2D U-Net Convolutional Neural Networks for Automated Cartilage and Meniscus Segmentation of Knee MR Imaging Data to Determine Relaxometry and Morphometry. , 2018, Radiology.

[47]  Stephen Lin,et al.  DeepVessel: Retinal Vessel Segmentation via Deep Learning and Conditional Random Field , 2016, MICCAI.

[48]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  George Azzopardi,et al.  Trainable COSFIRE filters for vessel delineation with application to retinal images , 2015, Medical Image Anal..

[50]  Hao Chen,et al.  3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes , 2016, MICCAI.

[51]  Veska Georgieva,et al.  Spleen segmentation in MRI sequence images using template matching and active contours , 2018 .

[52]  Joachim M. Buhmann,et al.  Crowdsourcing the creation of image segmentation algorithms for connectomics , 2015, Front. Neuroanat..

[53]  Li Fan,et al.  Lung Lesion Extraction Using a Toboggan Based Growing Automatic Segmentation Approach , 2016, IEEE Transactions on Medical Imaging.

[54]  Nasir M. Rajpoot,et al.  Dual-Channel Active Contour Model for Megakaryocytic Cell Segmentation in Bone Marrow Trephine Histology Images , 2017, IEEE Transactions on Biomedical Engineering.

[55]  Peng Liu,et al.  DeepDisc: Optic Disc Segmentation Based on Atrous Convolution and Spatial Pyramid Pooling , 2018, COMPAY/OMIA@MICCAI.

[56]  Dong Liu,et al.  Fully Convolutional Adaptation Networks for Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[57]  Qianjin Feng,et al.  Brain Tumor Segmentation Based on Local Independent Projection-Based Classification , 2014, IEEE Transactions on Biomedical Engineering.

[58]  Ke Chen,et al.  Automated Vessel Segmentation Using Infinite Perimeter Active Contour Model with Hybrid Region Information with Application to Retinal Images , 2015, IEEE Transactions on Medical Imaging.

[59]  Tien Yin Wong,et al.  Optic disc region of interest localization in fundus image for Glaucoma detection in ARGALI , 2010, 2010 5th IEEE Conference on Industrial Electronics and Applications.

[60]  Iman Aganj,et al.  Unsupervised Medical Image Segmentation Based on the Local Center of Mass , 2018, Scientific Reports.

[61]  Brahim Ait Skourt,et al.  Lung CT Image Segmentation Using Deep Neural Networks , 2018 .

[62]  Laude,et al.  FEEDBACK ON A PUBLICLY DISTRIBUTED IMAGE DATABASE: THE MESSIDOR DATABASE , 2014 .

[63]  Yuming Jiang,et al.  Optic Disc and Cup Segmentation with Blood Vessel Removal from Fundus Images for Glaucoma Detection , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[64]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[65]  Hiroshi Fujita,et al.  Automated detection of pulmonary nodules in helical CT images based on an improved template-matching technique , 2001, IEEE Transactions on Medical Imaging.

[66]  Richard Socher,et al.  Improving Generalization Performance by Switching from Adam to SGD , 2017, ArXiv.

[67]  M. Kanimozhi,et al.  Brain MR Image Segmentation Using Self Organizing Map , 2013 .

[68]  Manuel Emilio Gegúndez-Arias,et al.  Detecting the Optic Disc Boundary in Digital Fundus Images Using Morphological, Edge Detection, and Feature Extraction Techniques , 2010, IEEE Transactions on Medical Imaging.

[69]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[70]  Ian D. Reid,et al.  RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[71]  Kaiming He,et al.  Rethinking ImageNet Pre-Training , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[72]  Max A. Viergever,et al.  Ridge-based vessel segmentation in color images of the retina , 2004, IEEE Transactions on Medical Imaging.

[73]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[74]  Tien Yin Wong,et al.  ORIGA-light: An online retinal fundus image database for glaucoma analysis and research , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[75]  Dean C. Barratt,et al.  Automatic Multi-Organ Segmentation on Abdominal CT With Dense V-Networks , 2018, IEEE Transactions on Medical Imaging.

[76]  Wei Shen,et al.  Learning from Experts: Developing Transferable Deep Features for Patient-Level Lung Cancer Prediction , 2016, MICCAI.

[77]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[78]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.