Pelvic Organ Segmentation Using Distinctive Curve Guided Fully Convolutional Networks

Accurate segmentation of pelvic organs (i.e., prostate, bladder, and rectum) from CT image is crucial for effective prostate cancer radiotherapy. However, it is a challenging task due to: 1) low soft tissue contrast in CT images and 2) large shape and appearance variations of pelvic organs. In this paper, we employ a two-stage deep learning-based method, with a novel distinctive curve-guided fully convolutional network (FCN), to solve the aforementioned challenges. Specifically, the first stage is for fast and robust organ detection in the raw CT images. It is designed as a coarse segmentation network to provide region proposals for three pelvic organs. The second stage is for fine segmentation of each organ, based on the region proposal results. To better identify those indistinguishable pelvic organ boundaries, a novel morphological representation, namely, distinctive curve, is also introduced to help better conduct the precise segmentation. To implement this, in this second stage, a multi-task FCN is initially utilized to learn the distinctive curve and the segmentation map separately and then combine these two tasks to produce accurate segmentation map. The final segmentation results of all three pelvic organs are generated by a weighted max-voting strategy. We have conducted exhaustive experiments on a large and diverse pelvic CT data set for evaluating our proposed method. The experimental results demonstrate that our proposed method is accurate and robust for this challenging segmentation task, by also outperforming the state-of-the-art segmentation methods.

[1]  Dinggang Shen,et al.  Spatial distribution and longitudinal development of deep cortical sulcal landmarks in infants , 2014, NeuroImage.

[2]  Yaozong Gao,et al.  Deformable MR Prostate Segmentation via Deep Feature Learning and Sparse Patch Matching , 2016, IEEE Transactions on Medical Imaging.

[3]  Dinggang Shen,et al.  Segmenting CT prostate images using population and patient-specific statistics for radiotherapy , 2010, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[4]  Yinghuan Shi,et al.  Does Manual Delineation only Provide the Side Information in CT Prostate Segmentation? , 2017, MICCAI.

[5]  Dorin Comaniciu,et al.  3D Deep Learning for Efficient and Robust Landmark Detection in Volumetric Data , 2015, MICCAI.

[6]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Yaozong Gao,et al.  Detecting Anatomical Landmarks for Fast Alzheimer’s Disease Diagnosis , 2016, IEEE Transactions on Medical Imaging.

[8]  Shu Liao,et al.  Automated bone segmentation from dental CBCT images using patch-based sparse representation and convex optimization. , 2014, Medical physics.

[9]  Dinggang Shen,et al.  Semi-supervised learning for pelvic MR image segmentation based on multi-task residual fully convolutional networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[10]  Tao Zhang,et al.  Model-based segmentation of medical imagery by matching distributions , 2005, IEEE Transactions on Medical Imaging.

[11]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[12]  Andrew Zisserman,et al.  Flowing ConvNets for Human Pose Estimation in Videos , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[13]  Yaozong Gao,et al.  Context-Aware Anatomical Landmark Detection: Application to Deformable Model Initialization in Prostate CT Images , 2014, MLMI.

[14]  Yaozong Gao,et al.  Fully convolutional networks for multi-modality isointense infant brain image segmentation , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[15]  Kuo Men,et al.  Automatic segmentation of the clinical target volume and organs at risk in the planning CT for rectal cancer using deep dilated convolutional neural networks , 2017, Medical physics.

[16]  Nathan Lay,et al.  Rapid Multi-organ Segmentation Using Context Integration and Discriminative Models , 2013, IPMI.

[17]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[18]  Pascal Haigron,et al.  Segmentation of pelvic structures for planning CT using a geometrical shape model tuned by a multi-scale edge detector , 2014, Physics in medicine and biology.

[19]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[20]  Yaozong Gao,et al.  Locally-constrained boundary regression for segmentation of prostate and rectum in the planning CT images , 2015, Medical Image Anal..

[21]  Yinghuan Shi,et al.  Semi-Automatic Segmentation of Prostate in CT Images via Coupled Feature Representation and Spatial-Constrained Transductive Lasso , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Siqi Chen,et al.  Segmenting the prostate and rectum in CT imagery using anatomical constraints , 2011, Medical Image Anal..

[23]  Lubomir M. Hadjiiski,et al.  Urinary bladder segmentation in CT urography using deep-learning convolutional neural network and level sets. , 2016, Medical physics.

[24]  Dinggang Shen,et al.  Automated Segmentation of 3D US Prostate Images Using Statistical Texture-Based Matching Method , 2003, MICCAI.

[25]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Dinggang Shen,et al.  Simulating deformations of MR brain images for validation of atlas-based segmentation and registration algorithms , 2006, NeuroImage.

[27]  Yinghuan Shi,et al.  Prostate Segmentation in CT Images via Spatial-Constrained Transductive Lasso , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Horst Bischof,et al.  Regressing Heatmaps for Multiple Landmark Localization Using CNNs , 2016, MICCAI.

[29]  Yaozong Gao,et al.  Accurate Segmentation of CT Male Pelvic Organs via Regression-Based Deformable Models and Multi-Task Random Forests , 2016, IEEE Transactions on Medical Imaging.

[30]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[31]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[32]  Ali Kamen,et al.  Constrained Surface Evolutions for Prostate and Bladder Segmentation in CT Images , 2005, CVBIA.

[33]  Xiaoou Tang,et al.  Facial Landmark Detection by Deep Multi-task Learning , 2014, ECCV.

[34]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[35]  David J. Hawkes,et al.  Deformable Vessel-Based Registration Using Landmark-Guided Coherent Point Drift , 2010, MIAR.

[36]  Chao Lu,et al.  Precise Segmentation of Multiple Organs in CT Volumes Using Learning-Based Approach and Information Theory , 2012, MICCAI.

[37]  Hervé Delingette,et al.  Automatic Segmentation of Bladder and Prostate Using Coupled 3D Deformable Models , 2007, MICCAI.

[38]  Yaozong Gao,et al.  Joint Learning of Image Regressor and Classifier for Deformable Segmentation of CT Pelvic Organs , 2015, MICCAI.