Multiregion segmentation of bladder cancer structures in MRI with progressive dilated convolutional networks

PURPOSE Precise segmentation of bladder walls and tumor regions is an essential step toward noninvasive identification of tumor stage and grade, which is critical for treatment decision and prognosis of patients with bladder cancer (BC). However, the automatic delineation of bladder walls and tumor in magnetic resonance images (MRI) is a challenging task, due to important bladder shape variations, strong intensity inhomogeneity in urine, and very high variability across the population, particularly on tumors' appearance. To tackle these issues, we propose to leverage the representation capacity of deep fully convolutional neural networks. METHODS The proposed network includes dilated convolutions to increase the receptive field without incurring extra cost or degrading its performance. Furthermore, we introduce progressive dilations in each convolutional block, thereby enabling extensive receptive fields without the need for large dilation rates. The proposed network is evaluated on 3.0T T2-weighted MRI scans from 60 pathologically confirmed patients with BC. RESULTS Experiments show the proposed model to achieve a higher level of accuracy than state-of-the-art methods, with a mean Dice similarity coefficient of 0.98, 0.84, and 0.69 for inner wall, outer wall, and tumor region segmentation, respectively. These results represent a strong agreement with reference contours and an increase in performance compared to existing methods. In addition, inference times are less than a second for a whole three-dimensional (3D) volume, which is between two and three orders of magnitude faster than related state-of-the-art methods for this application. CONCLUSION We showed that a CNN can yield precise segmentation of bladder walls and tumors in BC patients on MRI. The whole segmentation process is fully automatic and yields results similar to the reference standard, demonstrating the viability of deep learning models for the automatic multiregion segmentation of bladder cancer MRI images.

[1]  Yang Liu,et al.  Relationship between Glioblastoma Heterogeneity and Survival Time: An MR Imaging Texture Analysis , 2017, American Journal of Neuroradiology.

[2]  Zhengrong Liang,et al.  An Adaptive Window-Setting Scheme for Segmentation of Bladder Tumor Surface via MR Cystography , 2012, IEEE Transactions on Information Technology in Biomedicine.

[3]  Zhiming Luo,et al.  GridNet with Automatic Shape Prior Registration for Automatic MRI Cardiac Segmentation , 2017, STACOM@MICCAI.

[4]  Zhengrong Liang,et al.  Radiomics assessment of bladder cancer grade using texture features from diffusion‐weighted imaging , 2017, Journal of magnetic resonance imaging : JMRI.

[5]  Lubomir M. Hadjiiski,et al.  Bladder Cancer Segmentation in CT for Treatment Response Assessment: Application of Deep-Learning Convolution Neural Network—A Pilot Study , 2016, Tomography.

[6]  Seyed-Ahmad Ahmadi,et al.  Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields , 2016, MICCAI.

[7]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  P. Lambin,et al.  Radiomics: the bridge between medical imaging and personalized medicine , 2017, Nature Reviews Clinical Oncology.

[10]  Luis Miguel Bergasa,et al.  Efficient ConvNet for real-time semantic segmentation , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[11]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Keisuke Nemoto,et al.  Effective Use of Dilated Convolutions for Segmenting Small Object Instances in Remote Sensing Imagery , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[13]  Jonathan Ventura,et al.  Dilated Convolutions for Brain Tumor Segmentation in MRI Scans , 2017, BrainLes@MICCAI.

[14]  Gerhard Jakse,et al.  The updated EAU guidelines on muscle-invasive and metastatic bladder cancer. , 2009, European urology.

[15]  M. Knowles,et al.  Molecular biology of bladder cancer: new insights into pathogenesis and clinical diversity , 2014, Nature Reviews Cancer.

[16]  Guang Yang,et al.  Automatic Brain Tumor Detection and Segmentation Using U-Net Based Fully Convolutional Networks , 2017, MIUA.

[17]  Anton van den Hengel,et al.  High-performance Semantic Segmentation Using Very Deep Fully Convolutional Networks , 2016, ArXiv.

[18]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[19]  Marios Anthimopoulos,et al.  Semantic Segmentation of Pathological Lung Tissue With Dilated Fully Convolutional Networks , 2018, IEEE Journal of Biomedical and Health Informatics.

[20]  Zhiming Luo,et al.  Novel Deep Convolution Neural Network Applied to MRI Cardiac Segmentation , 2017, ArXiv.

[21]  Lena Gorelick,et al.  Convexity Shape Prior for Binary Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Zhengrong Liang,et al.  A new partial volume segmentation approach to extract bladder wall for computer-aided detection in virtual cystoscopy , 2004, SPIE Medical Imaging.

[23]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[24]  J. Hornaday,et al.  Cancer Facts & Figures 2004 , 2004 .

[25]  Benjamin Thyreau,et al.  Comparing fully automated state-of-the-art cerebellum parcellation from magnetic resonance images , 2018, NeuroImage.

[26]  Garrison W. Cottrell,et al.  Understanding Convolution for Semantic Segmentation , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[27]  Richard Kronland-Martinet,et al.  A real-time algorithm for signal analysis with the help of the wavelet transform , 1989 .

[28]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[29]  Steven J. M. Jones,et al.  Comprehensive molecular characterization of urothelial bladder carcinoma , 2014, Nature.

[30]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[31]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[32]  Tim Leiner,et al.  Dilated Convolutional Neural Networks for Cardiovascular MR Segmentation in Congenital Heart Disease , 2016, RAMBO+HVSMR@MICCAI.

[33]  Jacques Ferlay,et al.  Bladder Cancer Incidence and Mortality: A Global Overview and Recent Trends. , 2017, European urology.

[34]  H. Ozen.  Bladder cancer. , 1998, Current opinion in oncology.

[35]  Jose Dolz,et al.  3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study , 2016, NeuroImage.

[36]  J. Cho,et al.  Diagnostic performance of MRI for prediction of muscle-invasiveness of bladder cancer: A systematic review and meta-analysis. , 2017, European journal of radiology.

[37]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[39]  Zhengrong Liang,et al.  Segmentation of multispectral bladder MR images with inhomogeneity correction for virtual cystoscopy , 2008, SPIE Medical Imaging.

[40]  Lubomir M. Hadjiiski,et al.  Urinary bladder segmentation in CT urography using deep-learning convolutional neural network and level sets. , 2016, Medical physics.

[41]  Mitko Veta,et al.  Adversarial Training and Dilated Convolutions for Brain MRI Segmentation , 2017, DLMIA/ML-CDS@MICCAI.

[42]  Xuelong Li,et al.  Adaptive Shape Prior Constrained Level Sets for Bladder MR Image Segmentation , 2014, IEEE Journal of Biomedical and Health Informatics.

[43]  Jing Yuan,et al.  HyperDense-Net: A Hyper-Densely Connected CNN for Multi-Modal Image Segmentation , 2018, IEEE Transactions on Medical Imaging.

[44]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[45]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[46]  Jose Dolz,et al.  Unbiased Shape Compactness for Segmentation , 2017, MICCAI.

[47]  Dinggang Shen,et al.  Deep CNN ensembles and suggestive annotations for infant brain MRI segmentation , 2017, Comput. Medical Imaging Graph..

[48]  João Manuel R. S. Tavares,et al.  Novel Approach to Segment the Inner and Outer Boundaries of the Bladder Wall in T2-Weighted Magnetic Resonance Images , 2011, Annals of Biomedical Engineering.

[49]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Konstantinos Kamnitsas,et al.  Ensembles of Multiple Models and Architectures for Robust Brain Tumour Segmentation , 2017, BrainLes@MICCAI.

[52]  Tianshuai Liu,et al.  Three-dimensional texture features from intensity and high-order derivative maps for the discrimination between bladder tumors and wall tissues via MRI , 2017, International Journal of Computer Assisted Radiology and Surgery.

[53]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[54]  Pierrick Coupé,et al.  MRI white matter lesion segmentation using an ensemble of neural networks and overcomplete patch-based voting , 2018, Comput. Medical Imaging Graph..

[55]  L. R. Dice.  Measures of the Amount of Ecologic Association Between Species , 1945 .

[56]  Michael Brady,et al.  Segmentation of the bladder wall using coupled level set methods , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[57]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Hongbing Lu,et al.  Preoperative prediction of muscular invasiveness of bladder cancer with radiomic features on conventional MRI and its high-order derivative maps , 2017, Abdominal Radiology.

[59]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[60]  J. Bezdek,et al.  FCM: The fuzzy c-means clustering algorithm , 1984 .

[61]  Eugenio Culurciello,et al.  ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation , 2016, ArXiv.

[62]  The Cancer Genome Atlas Research Network,et al.  Comprehensive molecular characterization of urothelial bladder carcinoma , 2014, Nature.

[63]  Sotirios A. Tsaftaris,et al.  Medical Image Computing and Computer Assisted Intervention , 2017 .

[64]  Hao Chen,et al.  Gland segmentation in colon histology images: The glas challenge contest , 2016, Medical Image Anal..

[65]  Xi Zhang,et al.  3D detection and extraction of bladder tumors via MR virtual cystoscopy , 2015, International Journal of Computer Assisted Radiology and Surgery.

[66]  Zhengrong Liang,et al.  A Coupled Level Set Framework for Bladder Wall Segmentation With Application to MR Cystography , 2010, IEEE Transactions on Medical Imaging.

[67]  K. Baggerly,et al.  Identification of distinct basal and luminal subtypes of muscle-invasive bladder cancer with different sensitivities to frontline chemotherapy. , 2014, Cancer cell.

[68]  Jing Yuan,et al.  Simultaneous Segmentation of Multiple Regions in 3D Bladder MRI by Efficient Convex Optimization of Coupled Surfaces , 2017, ICIG.

[69]  Dimos Baltas,et al.  Esophagus segmentation in CT via 3D fully convolutional neural network and random walk , 2017, Medical physics.

[70]  Kuo Men,et al.  Automatic segmentation of the clinical target volume and organs at risk in the planning CT for rectal cancer using deep dilated convolutional neural networks , 2017, Medical physics.

[71]  Ke Wu,et al.  Bladder segmentation in MRI images using active region growing model , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[72]  Zhengrong Liang,et al.  A unified EM approach to bladder wall segmentation with coupled level-set constraints , 2013, Medical Image Anal..

[73]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[74]  J. Witjes,et al.  MUSCLE-INVASIVE AND METASTATIC BLADDER CANCER , 2016 .