Multi-Depth Fusion Network for Whole-Heart CT Image Segmentation

Obtaining precise whole-heart segmentation from computed tomography (CT) or other imaging techniques is prerequisite to clinically analyze the cardiac status, which plays an important role in the treatment of cardiovascular diseases. However, the whole-heart segmentation is still a challenging task due to the characteristic of medical images, such as far more background voxels than foreground voxels and the indistinct boundaries of adjacent tissues. In this paper, we first present a new deeply supervised 3D UNET which applies multi-depth fusion to the original network for a better extract context information. Then, we apply focal loss to the field of image segmentation and expand its application to multi-category tasks. Finally, the focal loss is incorporated into the Dice loss function (which can be used to solve category imbalance problem) to form a new loss function, which we call hybrid loss. We evaluate our new pipeline on the MICCAI 2017 whole-heart CT dataset, and it obtains a Dice score of 90.73%, which is better than most of the state-of-the-art methods.

[1]  Yading Yuan,et al.  Automatic Skin Lesion Segmentation Using Deep Fully Convolutional Networks With Jaccard Distance , 2017, IEEE Transactions on Medical Imaging.

[2]  Gregory Shakhnarovich,et al.  FractalNet: Ultra-Deep Neural Networks without Residuals , 2016, ICLR.

[3]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Xin Yang,et al.  3D Convolutional Networks for Fully Automatic Fine-Grained Whole Heart Partition , 2017, STACOM@MICCAI.

[5]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[6]  Saeid Nahavandi,et al.  Multi-Residual Networks: Improving the Speed and Accuracy of Residual Networks , 2016, 1609.05672.

[7]  Serge J. Belongie,et al.  Residual Networks Behave Like Ensembles of Relatively Shallow Networks , 2016, NIPS.

[8]  Hao Chen,et al.  VoxResNet: Deep voxelwise residual networks for brain segmentation from 3D MR images , 2017, NeuroImage.

[9]  Dahua Lin,et al.  PolyNet: A Pursuit of Structural Diversity in Very Deep Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Horst Bischof,et al.  Multi-label Whole Heart Segmentation Using CNNs and Anatomical Label Configurations , 2017, STACOM@MICCAI.

[11]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Maria Wimmer,et al.  Deep Sequential Segmentation of Organs in Volumetric Medical Scans , 2018, IEEE Transactions on Medical Imaging.

[13]  Daniel Rueckert,et al.  Multiatlas whole heart segmentation of CT data using conditional entropy for atlas ranking and selection. , 2015, Medical physics.

[14]  Xiahai Zhuang,et al.  Challenges and methodologies of fully automatic whole heart segmentation: a review. , 2013, Journal of healthcare engineering.

[15]  Örjan Smedby,et al.  Automatic Whole Heart Segmentation Using Deep Learning and Shape Context , 2017, STACOM@MICCAI.

[16]  Xiahai Zhuang,et al.  Multi-scale patch and multi-modality atlases for whole heart segmentation of MRI , 2016, Medical Image Anal..

[17]  Daniel Rueckert,et al.  Hierarchical Statistical Shape Analysis and Prediction of Sub-cortical Brain Structures , 2007 .

[18]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Trevor Darrell,et al.  Deep Layer Aggregation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[21]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[22]  Yong Fan,et al.  A deep learning model integrating FCNNs and CRFs for brain tumor segmentation , 2017, Medical Image Anal..

[23]  Daniel Rueckert,et al.  Multi-atlas segmentation with augmented features for cardiac MR images , 2015, Medical Image Anal..

[24]  Jing Qin,et al.  3D Deeply-Supervised U-Net Based Whole Heart Segmentation , 2017, STACOM@MICCAI.

[25]  Jose Dolz,et al.  3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study , 2016, NeuroImage.

[26]  Dean C. Barratt,et al.  Automatic Multi-Organ Segmentation on Abdominal CT With Dense V-Networks , 2018, IEEE Transactions on Medical Imaging.

[27]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[28]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[30]  Jean-Yves Ramel,et al.  Local Probabilistic Atlases and a Posteriori Correction for the Segmentation of Heart Images , 2017, STACOM@MICCAI.

[31]  S. Dwivedi,et al.  Obesity May Be Bad: Compressed Convolutional Networks for Biomedical Image Segmentation , 2020 .

[32]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[33]  Zhuowen Tu,et al.  Deeply-Supervised Nets , 2014, AISTATS.

[34]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Jichao Zhao,et al.  Fully Automatic Left Atrium Segmentation From Late Gadolinium Enhanced Magnetic Resonance Imaging Using a Dual Fully Convolutional Neural Network , 2019, IEEE Transactions on Medical Imaging.

[36]  Timothy F. Cootes,et al.  3D Brain Segmentation Using Active Appearance Models and Local Regressors , 2008, MICCAI.

[37]  Mattias P. Heinrich,et al.  MRI Whole Heart Segmentation Using Discrete Nonlinear Registration and Fast Non-local Fusion , 2017, STACOM@MICCAI.

[38]  Ulas Bagci,et al.  Multi-Planar Deep Segmentation Networks for Cardiac Substructures from MRI and CT , 2017, STACOM@MICCAI.