Iterative Label Denoising Network: Segmenting Male Pelvic Organs in CT From 3D Bounding Box Annotations

Obtaining accurate segmentation of the prostate and nearby organs at risk (e.g., bladder and rectum) in CT images is critical for radiotherapy of prostate cancer. Currently, the leading automatic segmentation algorithms are based on Fully Convolutional Networks (FCNs), which achieve remarkable performance but usually need large-scale datasets with high-quality voxel-wise annotations for full supervision of the training. Unfortunately, such annotations are difficult to acquire, which becomes a bottleneck to build accurate segmentation models in real clinical applications. In this paper, we propose a novel weakly supervised segmentation approach that only needs 3D bounding box annotations covering the organs of interest to start the training. Obviously, the bounding box includes many non-organ voxels that carry noisy labels to mislead the segmentation model. To this end, we propose the label denoising module and embed it into the iterative training scheme of the label denoising network (LDnet) for segmentation. The labels of the training voxels are predicted by the tentative LDnet, while the label denoising module identifies the voxels with unreliable labels. As only the good training voxels are preserved, the iteratively re-trained LDnet can refine its segmentation capability gradually. Our results are remarkable, i.e., reaching <inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>94% (prostate), <inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>91% (bladder), and <inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>86% (rectum) of the Dice Similarity Coefficients (DSCs), compared to the case of fully supervised learning upon high-quality voxel-wise annotations and also superior to several state-of-the-art approaches. To our best knowledge, this is the first work to achieve voxel-wise segmentation in CT images from simple 3D bounding box annotations, which can greatly reduce many labeling efforts and meet the demands of the practical clinical applications.

[1]  Joachim M. Buhmann,et al.  Prostate MRI Segmentation Using Learned Semantic Knowledge and Graph Cuts , 2014, IEEE Transactions on Biomedical Engineering.

[2]  Frans Vos,et al.  Computer-Aided Detection of Polyps in CT Colonography Using Logistic Regression , 2010, IEEE Transactions on Medical Imaging.

[3]  Yadong Wang,et al.  Accurate Pelvis and Femur Segmentation in Hip CT With a Novel Patch-Based Refinement , 2019, IEEE Journal of Biomedical and Health Informatics.

[4]  Shenghua Gao,et al.  CE-Net: Context Encoder Network for 2D Medical Image Segmentation , 2019, IEEE Transactions on Medical Imaging.

[5]  Huaici Zhao,et al.  Computer aided endoscope diagnosis via weakly labeled data mining , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[6]  Sébastien Ourselin,et al.  DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Eric Granger,et al.  Constrained‐CNN losses for weakly supervised segmentation☆ , 2018, Medical Image Anal..

[8]  Zhipeng Jia,et al.  Constrained Deep Weak Supervision for Histopathology Image Segmentation , 2017, IEEE Transactions on Medical Imaging.

[9]  Jinjun Xiong,et al.  TS2C: Tight Box Mining with Surrounding Segmentation Context for Weakly Supervised Object Detection , 2018, ECCV.

[10]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[11]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[12]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Éloi Bossé,et al.  An iterative possibilistic knowledge diffusion approach for blind medical image segmentation , 2018, Pattern Recognit..

[14]  Liang Chen,et al.  DRINet for Medical Image Segmentation , 2018, IEEE Transactions on Medical Imaging.

[15]  Hao Chen,et al.  DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Elisa Ricci,et al.  Retinal Blood Vessel Segmentation Using Line Operators and Support Vector Classification , 2007, IEEE Transactions on Medical Imaging.

[17]  Konstantinos Kamnitsas,et al.  DeepCut: Object Segmentation From Bounding Box Annotations Using Convolutional Neural Networks , 2016, IEEE Transactions on Medical Imaging.

[18]  Bir Bhanu,et al.  Dynamic Low-Level Context for the Detection of Mild Traumatic Brain Injury , 2015, IEEE Transactions on Biomedical Engineering.

[19]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[20]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[21]  Luc Van Gool,et al.  Weakly Supervised Cascaded Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Abhinav Gupta,et al.  Learning from Noisy Large-Scale Datasets with Minimal Supervision , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Maryam Sabzevari,et al.  Vote-boosting ensembles , 2016, Pattern Recognit..

[24]  Aritra Ghosh,et al.  Robust Loss Functions under Label Noise for Deep Neural Networks , 2017, AAAI.

[25]  Yuri Boykov,et al.  Normalized Cut Loss for Weakly-Supervised CNN Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[27]  Xingrui Yu,et al.  Co-teaching: Robust training of deep neural networks with extremely noisy labels , 2018, NeurIPS.

[28]  Yinghuan Shi,et al.  Pelvic Organ Segmentation Using Distinctive Curve Guided Fully Convolutional Networks , 2019, IEEE Transactions on Medical Imaging.

[29]  Christian Wachinger,et al.  DeepNAT: Deep convolutional neural network for segmenting neuroanatomy , 2017, NeuroImage.

[30]  Dinggang Shen,et al.  Segmenting CT prostate images using population and patient-specific statistics for radiotherapy , 2010, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[31]  Yu Zhang,et al.  Exploit Bounding Box Annotations for Multi-Label Object Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Wenhao Huang,et al.  Dynamic boosting in deep learning using reconstruction error , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[33]  Yaozong Gao,et al.  CT male pelvic organ segmentation using fully convolutional networks with boundary sensitive representation , 2019, Medical Image Anal..

[34]  David Dagan Feng,et al.  Atlas registration and ensemble deep convolutional neural network-based prostate segmentation using magnetic resonance imaging , 2018, Neurocomputing.

[35]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[36]  J. Hornaday,et al.  Cancer Facts & Figures 2004 , 2004 .

[37]  Yoshua Bengio,et al.  The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[38]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Yaozong Gao,et al.  Learning Distance Transform for Boundary Detection and Deformable Segmentation in CT Prostate Images , 2014, MLMI.

[40]  Chao Lu,et al.  An integrated approach to segmentation and nonrigid registration for application in image-guided pelvic radiotherapy , 2011, Medical Image Anal..

[41]  Dinggang Shen,et al.  Automated Segmentation of 3D US Prostate Images Using Statistical Texture-Based Matching Method , 2003, MICCAI.

[42]  Bin Yang,et al.  Learning to Reweight Examples for Robust Deep Learning , 2018, ICML.

[43]  D. Mahapatra,et al.  Analyzing Training Information From Random Forests for Improved Image Segmentation , 2014, IEEE Transactions on Image Processing.

[44]  Jian Sun,et al.  BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[45]  Lars Petersson,et al.  Built-in Foreground/Background Prior for Weakly-Supervised Semantic Segmentation , 2016, ECCV.

[46]  Pravin Kakar,et al.  If You Can't Beat Them, Join Them: Learning with Noisy Data , 2015, ACM Multimedia.

[47]  Huaici Zhao,et al.  Computer-Aided Endoscopic Diagnosis Without Human-Specific Labeling , 2016, IEEE Transactions on Biomedical Engineering.

[48]  Yunchao Wei,et al.  STC: A Simple to Complex Framework for Weakly-Supervised Semantic Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  J. Friedman.  Stochastic gradient boosting , 2002 .

[50]  Yaozong Gao,et al.  Accurate Segmentation of CT Male Pelvic Organs via Regression-Based Deformable Models and Multi-Task Random Forests , 2016, IEEE Transactions on Medical Imaging.

[51]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[52]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[53]  Luc Van Gool,et al.  Deep Extreme Cut: From Extreme Points to Object Segmentation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[54]  Bernt Schiele,et al.  Simple Does It: Weakly Supervised Instance and Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Maoguo Gong,et al.  RBoost: Label Noise-Robust Boosting Algorithm Based on a Nonconvex Loss Function and the Numerically Stable Base Learners , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[56]  Yaozong Gao,et al.  Locally-constrained boundary regression for segmentation of prostate and rectum in the planning CT images , 2015, Medical Image Anal..

[57]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Christoph Meinel,et al.  Deep Learning for Medical Image Analysis , 2018, Journal of Pathology Informatics.