Automatic magnetic resonance prostate segmentation by deep learning with holistically nested networks

Abstract. Accurate automatic segmentation of the prostate in magnetic resonance images (MRI) is a challenging task due to the high variability of prostate anatomic structure. Artifacts such as noise and similar signal intensity of tissues around the prostate boundary inhibit traditional segmentation methods from achieving high accuracy. We investigate both patch-based and holistic (image-to-image) deep-learning methods for segmentation of the prostate. First, we introduce a patch-based convolutional network that aims to refine the prostate contour which provides an initialization. Second, we propose a method for end-to-end prostate segmentation by integrating holistically nested edge detection with fully convolutional networks. Holistically nested networks (HNN) automatically learn a hierarchical representation that can improve prostate boundary detection. Quantitative evaluation is performed on the MRI scans of 250 patients in fivefold cross-validation. The proposed enhanced HNN model achieves a mean ± standard deviation. A Dice similarity coefficient (DSC) of 89.77%±3.29% and a mean Jaccard similarity coefficient (IoU) of 81.59%±5.18% are used to calculate without trimming any end slices. The proposed holistic model significantly (p<0.001) outperforms a patch-based AlexNet model by 9% in DSC and 13% in IoU. Overall, the method achieves state-of-the-art performance as compared with other MRI prostate segmentation methods in the literature.

[1]  M. Habes,et al.  Automated prostate segmentation in whole-body MRI scans for epidemiological studies , 2013, Physics in medicine and biology.

[2]  Nacim Betrouni,et al.  Gland and Zonal Segmentation of Prostate on T2W MR Images , 2016, Journal of Digital Imaging.

[3]  Ferdi van der Heijden,et al.  A new prostate segmentation approach using multispectral magnetic resonance imaging and a statistical pattern classifier , 2012, Medical Imaging.

[4]  Bjarne K. Ersbøll,et al.  FAME-a flexible appearance modeling environment , 2003, IEEE Transactions on Medical Imaging.

[5]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[6]  Ronald M. Summers,et al.  Active appearance model and deep learning for more accurate prostate segmentation on MRI , 2016, SPIE Medical Imaging.

[7]  Florian Jung,et al.  Evaluation of prostate segmentation algorithms for MRI: The PROMISE12 challenge , 2014, Medical Image Anal..

[8]  Matthew J. McAuliffe,et al.  Medical Image Processing, Analysis and Visualization in clinical research , 2001, Proceedings 14th IEEE Symposium on Computer-Based Medical Systems. CBMS 2001.

[9]  Stefan Klein,et al.  Automatic segmentation of the prostate in 3D MR images by atlas matching using localized mutual information. , 2008, Medical physics.

[10]  Ronald M. Summers,et al.  Automatic Lymph Node Cluster Segmentation Using Holistically-Nested Neural Networks and Structured Optimization in CT Images , 2016, MICCAI.

[11]  Jesper Carl,et al.  The use of atlas registration and graph cuts for prostate segmentation in magnetic resonance images. , 2015, Medical physics.

[12]  Joachim Weickert,et al.  Coherence-Enhancing Diffusion Filtering , 1999, International Journal of Computer Vision.

[13]  Anant Madabhushi,et al.  Multifeature Landmark-Free Active Appearance Models: Application to Prostate MRI Segmentation , 2012, IEEE Transactions on Medical Imaging.

[14]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Hao Chen,et al.  Volumetric ConvNets with Mixed Residual Connections for Automated Prostate Segmentation from 3D MR Images , 2017, AAAI.

[16]  Fang Zhang,et al.  Deep convolutional activation features for large scale Brain Tumor histopathology image classification and segmentation , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[17]  Fabrice Mériaudeau,et al.  Texture Guided Active Appearance Model Propagation for Prostate Segmentation , 2010, Prostate Cancer Imaging.

[18]  Allan Hanbury,et al.  Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool , 2015, BMC Medical Imaging.

[19]  Yaozong Gao,et al.  Deformable MR Prostate Segmentation via Deep Feature Learning and Sparse Patch Matching , 2016, IEEE Transactions on Medical Imaging.

[20]  Baris Turkbey,et al.  Fully automated prostate segmentation in 3D MR based on normalized gradient fields cross-correlation initialization and LOGISMOS refinement , 2012, Medical Imaging.

[21]  Shu Liao,et al.  Representation Learning: A Unified Deep Learning Framework for Automatic Prostate MR Segmentation , 2013, MICCAI.

[22]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.