A novel MRI segmentation method using CNN‐based correction network for MRI‐guided adaptive radiotherapy

PURPOSE The purpose of this study was to expedite the contouring process for MRI-guided adaptive radiotherapy (MR-IGART), a convolutional neural network (CNN) deep-learning (DL) model is proposed to accurately segment the liver, kidneys, stomach, bowel and duodenum in 3D MR images. METHODS Images and structure contours for 120 patients were collected retrospectively. Treatment sites included pancreas, liver, stomach, adrenal gland, and prostate. The proposed DL model contains a voxel-wise label prediction CNN and a correction network which consists of two sub-networks. The prediction CNN and sub-networks in the correction network each includes a dense block which consists of twelve densely connected convolutional layers. The correction network was designed to improve the voxel-wise labeling accuracy of a CNN by learning and enforcing implicit anatomical constraints in the segmentation process. Its sub-networks learn to fix the erroneous classification of its previous network by taking as input both the original images and the softmax probability maps generated from its previous sub-network. The parameters of each sub-network were trained independently using piecewise training. The model was trained on 100 datasets, validated on 10 datasets and tested on the remaining 10 datasets. Dice coefficient, Hausdorff distance (HD) were calculated to evaluate the segmentation accuracy. RESULTS The proposed DL model was able to segment the organs with good accuracy. The correction network outperformed the conditional random field (CRF), a most comparable method that is usually applied as a post-processing step. For the 10 testing patients, the average Dice coefficients were 95.3 ± 0.73, 93.1 ± 2.22, 85.0 ± 3.75, 86.6 ± 2.69, and 65.5 ± 8.90 for liver, kidneys, stomach, bowel, and duodenum, respectively. The mean Hausdorff Distance (HD) were 5.41 ± 2.34, 6.23 ± 4.59, 6.88 ± 4.89, 5.90 ± 4.05, and 7.99 ± 6.84 mm, respectively. Manual contouring, as to correct the automatic segmentation results, was four times as fast as manual contouring from scratch. CONCLUSION The proposed method can automatically segment the liver, kidneys, stomach, bowel, and duodenum in 3D MR images with good accuracy. It is useful to expedite the manual contouring for MR-IGART.

[1]  Fang Lu,et al.  Automatic 3D liver location and segmentation via convolutional neural network and graph cut , 2016, International Journal of Computer Assisted Radiology and Surgery.

[2]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Sasa Mutic,et al.  The ViewRay system: magnetic resonance-guided and controlled radiotherapy. , 2014, Seminars in radiation oncology.

[4]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[5]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[6]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[7]  Hao Chen,et al.  VoxResNet: Deep voxelwise residual networks for brain segmentation from 3D MR images , 2017, NeuroImage.

[8]  Paolo Zaffino,et al.  Deep Neural Networks for Fast Segmentation of 3D Medical Images , 2016, MICCAI.

[9]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Sebastian Nowozin,et al.  Stabilizing Training of Generative Adversarial Networks through Regularization , 2017, NIPS.

[11]  Jose Dolz,et al.  3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study , 2016, NeuroImage.

[12]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[13]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[14]  Guosheng Lin,et al.  Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Sasa Mutic,et al.  Benchmark IMRT evaluation of a Co-60 MRI-guided radiation therapy system. , 2015, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[16]  Jialin Peng,et al.  Automatic 3D liver segmentation based on deep learning and globally optimized surface evolution , 2016, Physics in medicine and biology.

[17]  Xiangrong Zhou,et al.  Deep learning of the sectional appearances of 3D CT images for anatomical structure segmentation based on an FCN voting method , 2017, Medical physics.

[18]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[19]  Atsuto Maki,et al.  A systematic study of the class imbalance problem in convolutional neural networks , 2017, Neural Networks.

[20]  Seyed-Ahmad Ahmadi,et al.  Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields , 2016, MICCAI.

[21]  Parag J Parikh,et al.  Segmentation precision of abdominal anatomy for MRI-based radiotherapy. , 2014, Medical dosimetry : official journal of the American Association of Medical Dosimetrists.

[22]  Tao Xu,et al.  SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation , 2017, Neuroinformatics.

[23]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[24]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[25]  Lin Yang,et al.  An Automatic Learning-Based Framework for Robust Nucleus Segmentation , 2016, IEEE Transactions on Medical Imaging.

[26]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[27]  Sasa Mutic,et al.  Quality of Intensity Modulated Radiation Therapy Treatment Plans Using a ⁶⁰Co Magnetic Resonance Image Guidance Radiation Therapy System. , 2015, International journal of radiation oncology, biology, physics.

[28]  Max A. Viergever,et al.  Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities , 2016, MICCAI.

[29]  Minsong Cao,et al.  Online Adaptive Radiation Therapy: Implementation of a New Process of Care , 2017, Cureus.

[30]  Lin Yang,et al.  Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[31]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[32]  Thomas Brox,et al.  Generating Images with Perceptual Similarity Metrics based on Deep Networks , 2016, NIPS.

[33]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[34]  M. Viergever,et al.  Automatic Segmentation of MR Brain Images With a Convolutional Neural Network. , 2016, IEEE transactions on medical imaging.

[35]  Hao Chen,et al.  Automatic 3D Cardiovascular MR Segmentation with Densely-Connected Volumetric ConvNets , 2017, MICCAI.

[36]  Hao Chen,et al.  3D deeply supervised network for automated segmentation of volumetric medical images , 2017, Medical Image Anal..

[37]  Sebastian Nowozin,et al.  Which Training Methods for GANs do actually Converge? , 2018, ICML.

[38]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..