Deeply supervised 3D fully convolutional networks with group dilated convolution for automatic MRI prostate segmentation

PURPOSE Reliable automated segmentation of the prostate is indispensable for image-guided prostate interventions. However, the segmentation task is challenging due to inhomogeneous intensity distributions, variation in prostate anatomy, among other problems. Manual segmentation can be time-consuming and is subject to inter- and intraobserver variation. We developed an automated deep learning-based method to address this technical challenge. METHODS We propose a three-dimensional (3D) fully convolutional networks (FCN) with deep supervision and group dilated convolution to segment the prostate on magnetic resonance imaging (MRI). In this method, a deeply supervised mechanism was introduced into a 3D FCN to effectively alleviate the common exploding or vanishing gradients problems in training deep models, which forces the update process of the hidden layer filters to favor highly discriminative features. A group dilated convolution which aggregates multiscale contextual information for dense prediction was proposed to enlarge the effective receptive field of convolutional neural networks, which improve the prediction accuracy of prostate boundary. In addition, we introduced a combined loss function including cosine and cross entropy, which measures similarity and dissimilarity between segmented and manual contours, to further improve the segmentation accuracy. Prostate volumes manually segmented by experienced physicians were used as a gold standard against which our segmentation accuracy was measured. RESULTS The proposed method was evaluated on an internal dataset comprising 40 T2-weighted prostate MR volumes. Our method achieved a Dice similarity coefficient (DSC) of 0.86 ± 0.04, a mean surface distance (MSD) of 1.79 ± 0.46 mm, 95% Hausdorff distance (95%HD) of 7.98 ± 2.91 mm, and absolute relative volume difference (aRVD) of 15.65 ± 10.82. A public dataset (PROMISE12) including 50 T2-weighted prostate MR volumes was also employed to evaluate our approach. Our method yielded a DSC of 0.88 ± 0.05, MSD of 1.02 ± 0.35 mm, 95% HD of 9.50 ± 5.11 mm, and aRVD of 8.93 ± 7.56. CONCLUSION We developed a novel deeply supervised deep learning-based approach with a group dilated convolution to automatically segment the MRI prostate, demonstrated its clinical feasibility, and validated its accuracy against manual segmentation. The proposed technique could be a useful tool for image-guided interventions in prostate cancer.

[1]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Mark H. F. Savenije,et al.  Clinical implementation of MRI-based organs-at-risk auto-segmentation with convolutional networks for prostate radiotherapy , 2020, Radiation Oncology.

[3]  Josien P. W. Pluim,et al.  Patient Specific Prostate Segmentation in 3-D Magnetic Resonance Images , 2012, IEEE Transactions on Medical Imaging.

[4]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[5]  Ronald M. Summers,et al.  Active appearance model and deep learning for more accurate prostate segmentation on MRI , 2016, SPIE Medical Imaging.

[6]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[7]  Shu Liao,et al.  Representation Learning: A Unified Deep Learning Framework for Automatic Prostate MR Segmentation , 2013, MICCAI.

[8]  David Dagan Feng,et al.  Prostate segmentation in MR images using ensemble deep convolutional neural networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[9]  Tai-Hua Yang,et al.  Segmentation of finger tendon and synovial sheath in ultrasound image using deep convolutional neural network , 2020, Biomedical engineering online.

[10]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[11]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[12]  Hao Chen,et al.  3D deeply supervised network for automated segmentation of volumetric medical images , 2017, Medical Image Anal..

[13]  Yaozong Gao,et al.  LINKS: Learning-based multi-source IntegratioN frameworK for Segmentation of infant brain images , 2014, NeuroImage.

[14]  Yaozong Gao,et al.  Accurate Segmentation of CT Male Pelvic Organs via Regression-Based Deformable Models and Multi-Task Random Forests , 2016, IEEE Transactions on Medical Imaging.

[15]  Yaozong Gao,et al.  Deformable MR Prostate Segmentation via Deep Feature Learning and Sparse Patch Matching , 2017, Deep Learning for Medical Image Analysis.

[16]  Florian Jung,et al.  Evaluation of prostate segmentation algorithms for MRI: The PROMISE12 challenge , 2014, Medical Image Anal..

[17]  Anant Madabhushi,et al.  Multifeature Landmark-Free Active Appearance Models: Application to Prostate MRI Segmentation , 2012, IEEE Transactions on Medical Imaging.

[18]  Bo Du,et al.  Deeply-supervised CNN for prostate segmentation , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[19]  Christopher G Filippi,et al.  Applications of Artificial Intelligence to Prostate Multiparametric MRI (mpMRI): Current and Emerging Trends , 2020, Cancers.

[20]  Gustavo Carneiro,et al.  Deep Learning and Structured Prediction for the Segmentation of Mass in Mammograms , 2015, MICCAI.

[21]  Jesper Carl,et al.  The use of atlas registration and graph cuts for prostate segmentation in magnetic resonance images. , 2015, Medical physics.

[22]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[24]  Hao Jiang,et al.  Densely Dilated Spatial Pooling Convolutional Network using benign loss functions for imbalanced volumetric prostate segmentation , 2018 .

[25]  Baowei Fei,et al.  3D prostate segmentation of ultrasound images combining longitudinal image registration and machine learning , 2012, Medical Imaging.

[26]  David J. Kriegman,et al.  Dense Volume-to-Volume Vascular Boundary Detection , 2016, MICCAI.

[27]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[28]  Hao Chen,et al.  Volumetric ConvNets with Mixed Residual Connections for Automated Prostate Segmentation from 3D MR Images , 2017, AAAI.

[29]  Yang Lei,et al.  Male pelvic multi-organ segmentation aided by CBCT-based synthetic MRI , 2019, Physics in medicine and biology.

[30]  Tian Liu,et al.  3D transrectal ultrasound (TRUS) prostate segmentation based on optimal feature learning framework , 2016, SPIE Medical Imaging.

[31]  Yaozong Gao,et al.  Deformable segmentation of 3D MR prostate images via distributed discriminative dictionary and ensemble learning. , 2014, Medical physics.

[32]  Aaron Fenster,et al.  Automatic 3D segmentation of ultrasound images using atlas registration and statistical texture prior , 2011, Medical Imaging.

[33]  Yang Lei,et al.  MRI-Based Synthetic CT for Radiation Treatment of Prostate Cancer , 2018, International Journal of Radiation Oncology*Biology*Physics.

[34]  Yaozong Gao,et al.  LINKS: Learning-based multi-source IntegratioN frameworK for Segmentation of infant brain images , 2015, NeuroImage.

[35]  Xuelong Li,et al.  Prostate Segmentation in MR Images Using Discriminant Boundary Features , 2013, IEEE Transactions on Biomedical Engineering.

[36]  Xiuying Wang,et al.  A combinatorial Bayesian and Dirichlet model for prostate MR image segmentation using probabilistic image features , 2016, Physics in medicine and biology.

[37]  Soumya Ghose,et al.  A Random Forest Based Classification Approach to Prostate Segmentation in MRI , 2012 .

[38]  Garrison W. Cottrell,et al.  Understanding Convolution for Semantic Segmentation , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[39]  Zhiqiang Tian,et al.  A fully automatic multi-atlas based segmentation method for prostate MR images , 2015, Medical Imaging.

[40]  Hao Chen,et al.  Iterative Multi-domain Regularized Deep Learning for Anatomical Structure Detection and Segmentation from Ultrasound Images , 2016, MICCAI.

[41]  C A T van den Berg,et al.  Fast contour propagation for MR‐guided prostate radiotherapy using convolutional neural networks , 2019, Medical physics.

[42]  A. Jemal,et al.  International variation in prostate cancer incidence and mortality rates. , 2012, European urology.

[43]  Lin Yang,et al.  An Automatic Learning-Based Framework for Robust Nucleus Segmentation , 2016, IEEE Transactions on Medical Imaging.

[44]  Xiaofeng Yang,et al.  Patch-based label fusion for automatic multi-atlas-based prostate segmentation in MR images , 2016, SPIE Medical Imaging.

[45]  Fabio Martínez,et al.  A multiresolution prostate representation for automatic segmentation in magnetic resonance images , 2017, Medical physics.

[46]  Nico Karssemeijer,et al.  Computer-Aided Detection of Prostate Cancer in MRI , 2014, IEEE Transactions on Medical Imaging.

[47]  Tian Liu,et al.  Magnetic resonance imaging-based pseudo computed tomography using anatomic signature and joint dictionary learning , 2018, Journal of medical imaging.

[48]  Shu Liao,et al.  Sparse Patch-Based Label Propagation for Accurate Prostate Localization in CT Images , 2013, IEEE Transactions on Medical Imaging.

[49]  Chang-Su Kim,et al.  Comparison of objective functions in CNN-based prostate magnetic resonance image segmentation , 2017, 2017 IEEE International Conference on Image Processing (ICIP).