Attentionanatomy: A Unified Framework for Whole-Body Organs at Risk Segmentation Using Multiple Partially Annotated Datasets

Organs-at-risk (OAR) delineation in computed tomography (CT) is an important step in Radiation Therapy (RT) planning. Recently, deep learning based methods for OAR delineation have been proposed and applied in clinical practice for separate regions of the human body (head and neck, thorax, and abdomen). However, there are few researches regarding the end-to-end whole-body OARs delineation because the existing datasets are mostly partially or incompletely annotated for such task. In this paper, our proposed end-to-end convolutional neural network model, called AttentionAnatomy, can be jointly trained with three partially annotated datasets, segmenting OARs from whole body. Our main contributions are: 1) an attention module implicitly guided by body region label to modulate the segmentation branch output; 2) a prediction re-calibration operation, exploiting prior information of the input images, to handle partial-annotation(HPA) problem; 3) a new hybrid loss function combining batch Dice loss and spatially balanced focal loss to alleviate the organ size imbalance problem. Experimental results of our proposed framework presented significant improvements in both Sørensen-Dice coefficient (DSC) and 95% Hausdorff distance compared to the baseline model.

[1]  Deniz Erdogmus,et al.  Tversky Loss Function for Image Segmentation Using 3D Fully Convolutional Deep Networks , 2017, MLMI@MICCAI.

[2]  Yaozong Gao,et al.  Hierarchical Vertex Regression-Based Segmentation of Head and Neck CT Images for Radiotherapy Planning , 2018, IEEE Transactions on Image Processing.

[3]  Geraint Rees,et al.  Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy , 2018, ArXiv.

[4]  Blake Hannaford,et al.  Evaluation of segmentation methods on head and neck CT: Auto‐segmentation challenge 2015 , 2017, Medical physics.

[5]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[7]  Bulat Ibragimov,et al.  Segmentation of parotid glands from registered CT and MR images. , 2018, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[8]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Bulat Ibragimov,et al.  Segmentation of organs‐at‐risks in head and neck CT images using convolutional neural networks , 2017, Medical physics.

[10]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[11]  Hao Tang,et al.  NoduleNet: Decoupled False Positive Reductionfor Pulmonary Nodule Detection and Segmentation , 2019, MICCAI.

[12]  Dinggang Shen,et al.  Interleaved 3D-CNNs for joint segmentation of small-volume structures in head and neck CT images. , 2018, Medical physics.

[13]  Jayaram K. Udupa,et al.  Image quality and segmentation , 2018, Medical Imaging.

[14]  Jayaram K. Udupa,et al.  Hierarchical model-based object localization for auto-contouring in head and neck radiation therapy planning , 2018, Medical Imaging.

[15]  X. Jiang,et al.  Automated delineation of organs-at-risk in head and neck CT images using multi-output support vector regression , 2018, Medical Imaging.

[16]  Paolo Zaffino,et al.  Deep Neural Networks for Fast Segmentation of 3D Medical Images , 2016, MICCAI.

[17]  Hao Tang,et al.  Automatic Pulmonary Lobe Segmentation Using Deep Learning , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[18]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Xiaohui Xie,et al.  Clinically applicable deep learning framework for organs at risk delineation in CT images , 2019, Nature Machine Intelligence.

[20]  Adam Herout,et al.  Segmentation of Head and Neck Organs at Risk Using CNN with Batch Dice Loss , 2018, GCPR.

[21]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Jayaram K. Udupa,et al.  Auto-contouring via automatic anatomy recognition of organs at risk in head and neck cancer on CT images , 2018, Medical Imaging.

[23]  Xiaohui Xie,et al.  AnatomyNet: Deep learning for fast and fully automated whole‐volume segmentation of head and neck anatomy , 2018, Medical physics.