ANU-Net: Attention-based nested U-Net to exploit full resolution features for medical image segmentation

Abstract Organ cancer have a high mortality rate. In order to help doctors diagnose and treat organ lesion, an automatic medical image segmentation model is urgently needed as manually segmentation is time-consuming and error-prone. However, automatic segmentation of target organ from medical images is a challenging task because of organ’s uneven and irregular shapes. In this paper, we propose an attention-based nested segmentation network, named ANU-Net. Our proposed network has a deep supervised encoder-decoder architecture and a redesigned dense skip connection. ANU-Net introduces attention mechanism between nested convolutional blocks so that the features extracted at different levels can be merged with a task-related selection. Besides, we redesign a hybrid loss function combining with three kinds of losses to make full use of full resolution feature information. We evaluated proposed model on MICCAI 2017 Liver Tumor Segmentation (LiTS) Challenge Dataset and ISBI 2019 Combined Healthy Abdominal Organ Segmentation (CHAOS) Challenge. ANU-Net achieved very competitive performance for four kinds of medical image segmentation tasks.

[1]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Qingjie Liu,et al.  Road Extraction by Deep Residual U-Net , 2017, IEEE Geoscience and Remote Sensing Letters.

[4]  Ronan Collobert,et al.  Learning to Segment Object Candidates , 2015, NIPS.

[5]  Wenyu Liu,et al.  Deep patch learning for weakly supervised object classification and discovery , 2017, Pattern Recognit..

[6]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Nima Tajbakhsh,et al.  UNet++: A Nested U-Net Architecture for Medical Image Segmentation , 2018, DLMIA/ML-CDS@MICCAI.

[9]  Ronald M. Summers,et al.  DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation , 2015, MICCAI.

[10]  Hayit Greenspan,et al.  Fully Convolutional Network for Liver Segmentation and Lesions Detection , 2016, LABELS/DLMIA@MICCAI.

[11]  Yoshua Bengio,et al.  End-to-end attention-based large vocabulary speech recognition , 2015, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[13]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[14]  Wei Li,et al.  DeepUNet: A Deep Fully Convolutional Network for Pixel-Level Sea-Land Segmentation , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[15]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[16]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Peter Caccetta,et al.  ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[18]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Xiaoming Liu,et al.  Automatic segmentation of liver tumors from multiphase contrast-enhanced CT images based on FCNs , 2017, Artif. Intell. Medicine.

[20]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[21]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[22]  Zhuowen Tu,et al.  Deeply-Supervised Nets , 2014, AISTATS.

[23]  George Papandreou,et al.  MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Roberto Cipolla,et al.  Semantic object classes in video: A high-definition ground truth database , 2009, Pattern Recognit. Lett..

[25]  Horst Bischof,et al.  Multi-label Whole Heart Segmentation Using CNNs and Anatomical Label Configurations , 2017, STACOM@MICCAI.

[26]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Yongchao Gong,et al.  Mask Scoring R-CNN , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Jianxiong Xiao,et al.  SUN RGB-D: A RGB-D scene understanding benchmark suite , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[30]  Christian Igel,et al.  Deep Feature Learning for Knee Cartilage Segmentation Using a Triplanar Convolutional Neural Network , 2013, MICCAI.

[31]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[32]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[33]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[34]  Wenjun Zeng,et al.  Object Detection in Videos by High Quality Object Linking , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Danny Ziyi Chen,et al.  Detection of Glands and Villi by Collaboration of Domain Knowledge and Deep Learning , 2015, MICCAI.

[36]  Hao Chen,et al.  3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes , 2016, MICCAI.

[37]  Zhe Li,et al.  Evaluate the Malignancy of Pulmonary Nodules Using the 3-D Deep Leaky Noisy-OR Network , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[38]  Fang Lu,et al.  Automatic 3D liver location and segmentation via convolutional neural network and graph cut , 2016, International Journal of Computer Assisted Radiology and Surgery.

[39]  Wei An,et al.  Learning Parallax Attention for Stereo Image Super-Resolution , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Yi Yang,et al.  Attention to Scale: Scale-Aware Semantic Image Segmentation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Gabriel Eilertsen,et al.  HDR image reconstruction from a single exposure using deep CNNs , 2017, ACM Trans. Graph..

[42]  Michael Kistler,et al.  The Virtual Skeleton Database: An Open Access Repository for Biomedical Research and Collaboration , 2013, Journal of medical Internet research.

[43]  Chi-Wing Fu,et al.  H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes , 2018, IEEE Transactions on Medical Imaging.