ISDNet: Importance Guided Semi-supervised Adversarial Learning for Medical Image Segmentation

Recent deep neural networks have achieved great success in medical image segmentation. However, massive labeled training data should be provided during network training, which is time consuming with intensive labor work and even requires expertise knowledge. To address such challenge, inspired by typical GANs, we propose a novel end-to-end semi-supervised adversarial learning framework for medical image segmentation, called “Importance guided Semi-supervised Deep Networks” (ISDNet). While most existing works based on GANs use a classifier discriminator to achieve adversarial learning, we combine a fully convolutional discriminator and a classifier discriminator to fulfill better adversarial learning and self-taught learning. Specifically, we propose an importance weight network combined with our FCN-based confidence network, which can assist segmentation network to learn better local and global information. Extensive experiments are conducted on the LASC 2013 and the LiTS 2017 datasets to demonstrate the effectiveness of our approach.

[1]  Jian Sun,et al.  ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Xu Zhao,et al.  Segmentation of Left Atrium Through Combination of Deep Convolutional and Recurrent Neural Networks , 2018 .

[4]  Kaiming He,et al.  Group Normalization , 2018, ECCV.

[5]  Sébastien Ourselin,et al.  Multi-atlas Propagation Whole Heart Segmentation from MRI and CTA Using a Local Normalised Correlation Coefficient Criterion , 2013, FIMH.

[6]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[7]  Trevor Darrell,et al.  Constrained Convolutional Neural Networks for Weakly Supervised Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  Jean-Louis Dillenseger,et al.  Multi-atlas-Based Segmentation of the Left Atrium and Pulmonary Veins , 2013, STACOM.

[9]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[10]  Seong Joon Oh,et al.  Exploiting Saliency for Object Segmentation from Image Level Labels , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[12]  Jing Zhang,et al.  Importance Weighted Adversarial Nets for Partial Domain Adaptation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[13]  Hayit Greenspan,et al.  GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification , 2018, Neurocomputing.

[14]  Kawal S. Rhode,et al.  CardiacNET: Segmentation of Left Atrium and Proximal Pulmonary Veins from MRI Using Multi-view CNN , 2017, MICCAI.

[15]  Jeff Donahue,et al.  Large Scale GAN Training for High Fidelity Natural Image Synthesis , 2018, ICLR.

[16]  Shiguang Shan,et al.  Duplex Generative Adversarial Network for Unsupervised Domain Adaptation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Jian Sun,et al.  BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  Fei-Fei Li,et al.  What's the Point: Semantic Segmentation with Point Supervision , 2015, ECCV.

[19]  Jürgen Weese,et al.  Benchmark for Algorithms Segmenting the Left Atrium From 3D CT and MRI Datasets , 2015, IEEE Transactions on Medical Imaging.

[20]  Yaozong Gao,et al.  ASDNet: Attention Based Semi-supervised Deep Networks for Medical Image Segmentation , 2018, MICCAI.

[21]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[22]  Xi Peng,et al.  A Generative Adversarial Approach for Zero-Shot Learning from Noisy Texts , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[23]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.