Parasitic GAN for Semi-Supervised Brain Tumor Segmentation

In semantic segmentation, researchers face the shortage of pixel-level annotated data. And it is particularly severe in the medical images. On the other hand, the unlabeled data are abundantly produced in the diagnosis routine. In the paper, we introduced the Parasitic GAN for the brain tumor segmentation to exploit the unlabeled data more efficiently. Parasitic GAN is composed of three parts: the segmentor ${\mathcal{S}}$, the generator ${\mathcal{G}}$, and the discriminator ${\mathcal{D}}$. With the label maps produced by the segmentor and the supplementary label maps synthesized by the generator, the discriminator could learn a more precise boundary of ground truth. Thus, the segmentor benefits from the adversarial learning mechanism and the extra supervision provided by the discriminator. This parasitic relationship between the segmentor and the generative adversarial network (${\mathcal{G}}$ and ${\mathcal{D}}$) restricts the fitness ability of the segmentor and improves its generalization capacity. In practice, it definitely improved the performance of segmentor in brain tumor segmentation tasks, increasing the dice score 0.010-0.035.

[1]  Philip H. S. Torr,et al.  Weakly- and Semi-Supervised Panoptic Segmentation , 2022 .

[2]  Christos Davatzikos,et al.  Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features , 2017, Scientific Data.

[3]  Yu Liu,et al.  Transferable Semi-supervised Semantic Segmentation , 2017, AAAI.

[4]  Hao Chen,et al.  VoxResNet: Deep Voxelwise Residual Networks for Volumetric Brain Segmentation , 2016, ArXiv.

[5]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Alejandro F. Frangi,et al.  Multi-Input and Dataset-Invariant Adversarial Learning (MDAL) for Left and Right-Ventricular Coverage Estimation in Cardiac MRI , 2018, MICCAI.

[8]  Nassir Navab,et al.  Semi-supervised Deep Learning for Fully Convolutional Networks , 2017, MICCAI.

[9]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[10]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[11]  Yuichi Yoshida,et al.  Spectral Normalization for Generative Adversarial Networks , 2018, ICLR.

[12]  Jiajun Wu,et al.  Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling , 2016, NIPS.

[13]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[14]  C. V. Jawahar,et al.  Universal Semi-Supervised Semantic Segmentation , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[15]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[16]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[17]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[18]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[19]  Camille Couprie,et al.  Semantic Segmentation using Adversarial Networks , 2016, NIPS 2016.

[20]  Ming-Hsuan Yang,et al.  Adversarial Learning for Semi-supervised Semantic Segmentation , 2018, BMVC.

[21]  Yanchun Zhang,et al.  MDU-Net: multi-scale densely connected U-Net for biomedical image segmentation , 2018, Health Information Science and Systems.

[22]  Michael Kistler,et al.  The Virtual Skeleton Database: An Open Access Repository for Biomedical Research and Collaboration , 2013, Journal of medical Internet research.

[23]  Youbao Tang,et al.  CT-Realistic Lung Nodule Simulation from 3D Conditional Generative Adversarial Networks for Robust Lung Segmentation , 2018, MICCAI.

[24]  Guoyan Zheng,et al.  3D U-net with Multi-level Deep Supervision: Fully Automatic Segmentation of Proximal Femur in 3D MR Images , 2017, MLMI@MICCAI.

[25]  Jose Dolz,et al.  Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning , 2018, ArXiv.

[26]  Nima Tajbakhsh,et al.  UNet++: A Nested U-Net Architecture for Medical Image Segmentation , 2018, DLMIA/ML-CDS@MICCAI.