A new smart-cropping pipeline for prostate segmentation using deep learning networks

Abstract— Prostate segmentation from magnetic resonance imaging (MRI) is a challenging task. In recent years, several network architectures have been proposed to automate this process and alleviate the burden of manual annotation. Although the performance of these models has achieved promising results, there is still room for improvement before these models can be used safely and effectively in clinical practice. One of the major challenges in prostate MR image segmentation is the presence of class imbalance in the image labels where the background pixels dominate over the prostate. In the present work we propose a DL-based pipeline for cropping the region around the prostate from MRI images to produce a more balanced distribution of the foreground pixels (prostate) and the background pixels and improve segmentation accuracy. The effect of DL-cropping for improving the segmentation performance compared to standard centercropping is assessed using five popular DL networks for prostate segmentation, namely U-net, U-net+, Res Unet++, Bridge U-net and Dense U-net. The proposed smart-cropping outperformed the standard center cropping in terms of segmentation accuracy for all the evaluated prostate segmentation networks. In terms of Dice score, the highest improvement was achieved for the Unet+ and ResU-net++ architectures corresponding to 8.9% and 8%, respectively.

[1]  Kai Yang,et al.  A Deep Active Survival Analysis Approach for Precision Treatment Recommendations: Application of Prostate Cancer , 2018, Expert Syst. Appl..

[2]  Martial Hebert,et al.  Toward Objective Evaluation of Image Segmentation Algorithms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Jin Tae Kwak,et al.  Deep dense multi-path neural network for prostate segmentation in magnetic resonance imaging , 2018, International Journal of Computer Assisted Radiology and Surgery.

[4]  Francesco Visin,et al.  A guide to convolution arithmetic for deep learning , 2016, ArXiv.

[5]  Lawrence H. Staib,et al.  Learning Non‐rigid Deformations for Robust, Constrained Point‐based Registration in Image‐Guided MR‐TRUS Prostate Intervention , 2017, Medical Image Anal..

[6]  Khaled Alsaih,et al.  Evaluation of Deep Neural Networks for Semantic Segmentation of Prostate in T2W MRI , 2020, Sensors.

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Kevin Ho-Shon,et al.  Convolutional Neural Networks for Prostate Magnetic Resonance Image Segmentation , 2019, IEEE Access.

[9]  Arvid Lundervold,et al.  An overview of deep learning in medical imaging focusing on MRI , 2018, Zeitschrift fur medizinische Physik.

[10]  Thomas Martin Deserno,et al.  Survey: interpolation methods in medical image processing , 1999, IEEE Transactions on Medical Imaging.

[11]  Thomas de Lange,et al.  ResUNet++: An Advanced Architecture for Medical Image Segmentation , 2019, 2019 IEEE International Symposium on Multimedia (ISM).

[12]  Tianqi Chen,et al.  Empirical Evaluation of Rectified Activations in Convolutional Network , 2015, ArXiv.

[13]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Antonio J. Plaza,et al.  Image Segmentation Using Deep Learning: A Survey , 2021, IEEE transactions on pattern analysis and machine intelligence.

[15]  Xiongwei Wu,et al.  Recent Advances in Deep Learning for Object Detection , 2019, Neurocomputing.

[16]  Namkug Kim,et al.  Deep Learning in Medical Imaging , 2019, Neurospine.

[17]  Yinghuan Shi,et al.  An Effective MR-Guided CT Network Training for Segmenting Prostate in CT Images , 2019, IEEE Journal of Biomedical and Health Informatics.

[18]  Matthew B. Blaschko,et al.  Optimization for Medical Image Segmentation: Theory and Practice When Evaluating With Dice Score or Jaccard Index , 2020, IEEE Transactions on Medical Imaging.

[19]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[20]  M. Stone Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[21]  Joseph Paul Cohen,et al.  Deep semantic segmentation of natural and medical images: a review , 2019, Artificial Intelligence Review.

[22]  Christian Ledig,et al.  Is the deconvolution layer the same as a convolutional layer? , 2016, ArXiv.

[23]  Sven Behnke,et al.  Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition , 2010, ICANN.

[24]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[25]  A. Ouzzane,et al.  Magnetic Resonance Imaging Targeted Biopsy Improves Selection of Patients Considered for Active Surveillance for Clinically Low Risk Prostate Cancer Based on Systematic Biopsies. , 2015, The Journal of urology.

[26]  Ce Liu,et al.  Deep Convolutional Neural Network for Image Deconvolution , 2014, NIPS.

[27]  Florian Jung,et al.  Evaluation of prostate segmentation algorithms for MRI: The PROMISE12 challenge , 2014, Medical Image Anal..

[28]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  K. K. Sahu,et al.  Normalization: A Preprocessing Stage , 2015, ArXiv.

[30]  Evgin Goceri,et al.  Challenges and Recent Solutions for Image Segmentation in the Era of Deep Learning , 2019, 2019 Ninth International Conference on Image Processing Theory, Tools and Applications (IPTA).

[31]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[32]  Maria A Schmidt,et al.  Radiotherapy planning using MRI , 2015, Physics in medicine and biology.

[33]  Qingjie Liu,et al.  Road Extraction by Deep Residual U-Net , 2017, IEEE Geoscience and Remote Sensing Letters.

[34]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[36]  N. Barışçı,et al.  Prostate Segmentation via Fusing the Nested-V-net3d and V-net2d , 2019, 2019 1st International Informatics and Software Engineering Conference (UBMYK).

[37]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[38]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[39]  Matthew B. Blaschko,et al.  The Lovasz-Softmax Loss: A Tractable Surrogate for the Optimization of the Intersection-Over-Union Measure in Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[40]  L. R. Dice Measures of the Amount of Ecologic Association Between Species , 1945 .

[41]  Stephen M. Moore,et al.  The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository , 2013, Journal of Digital Imaging.

[42]  Stephen Marshall,et al.  Activation Functions: Comparison of trends in Practice and Research for Deep Learning , 2018, ArXiv.

[43]  Su Ruan,et al.  A review: Deep learning for medical image segmentation using multi-modality fusion , 2019, Array.

[44]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Yu Qiao,et al.  Prostate Segmentation using 2D Bridged U-net , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[46]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Craig H Meyer,et al.  Deep convolutional neural network for segmentation of thoracic organs-at-risk using cropped 3D images. , 2019, Medical physics.

[49]  Steven Guan,et al.  Fully Dense UNet for 2-D Sparse Photoacoustic Tomography Artifact Removal , 2018, IEEE Journal of Biomedical and Health Informatics.

[50]  Wojciech Czarnecki,et al.  On Loss Functions for Deep Neural Networks in Classification , 2017, ArXiv.

[51]  A. Jemal,et al.  Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries , 2021, CA: a cancer journal for clinicians.