Domain Adaptation based COVID-19 CT Lung Infections Segmentation Network

Coronavirus disease (COVID-19 pneumonia) has spread rapidly and become a global epidemic, which has had a great impact on public health and the economy. The automatic segmentation of lung infections from computed tomography (CT) has become an effective method for diagnosis. In order to realize the full potential of deep learning models in COVID-19 pneumonia infections segmentation, a great deal of annotated CT data is needed for training. The data is difficult to collect due to the high risk of infection, and it is laborious to annotate. Recent advances in image synthesis make it possible to train deep learning models on realistic synthetic data with computer-generated annotations. However, the domain shift between real data and synthetic data significantly reduces segmentation performance. In order to solve this issue, we propose a novel domain adaptation based COVID-19 CT lung infections segmentation network. In this work, we use limited real data without annotations and a large amount of annotated synthetic data to train the U-Net segmentation network jointly. To overcome the domain mismatch, we introduce conditional GAN for adversarial training. We update the segmentation network with the cross-domain adversarial loss. This makes the embedding distribution learned by segmentation network from real data and synthetic data closer, thus greatly improving the representation ability of the segmentation network. The experiment results demonstrate that our proposed network significantly outperforms the baseline and state-of-the-art methods.

[1]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[2]  Q. Tao,et al.  Serial Quantitative Chest CT Assessment of COVID-19: A Deep Learning Approach , 2020, Radiology. Cardiothoracic imaging.

[3]  Konstantinos Kamnitsas,et al.  Unsupervised domain adaptation in brain lesion segmentation with adversarial networks , 2016, IPMI.

[4]  Rama Chellappa,et al.  Visual Domain Adaptation: A survey of recent advances , 2015, IEEE Signal Processing Magazine.

[5]  Bernt Schiele,et al.  Generative Adversarial Text to Image Synthesis , 2016, ICML.

[6]  Xin Qi,et al.  Adversarial Domain Adaptation for Classification of Prostate Histopathology Whole-Slide Images , 2018, MICCAI.

[7]  Christian S. Perone,et al.  Unsupervised domain adaptation for medical imaging segmentation with self-ensembling , 2018, NeuroImage.

[8]  Z. Fayad,et al.  CT Imaging Features of 2019 Novel Coronavirus (2019-nCoV) , 2020, Radiology.

[9]  Nojun Kwak,et al.  MC-GAN: Multi-conditional Generative Adversarial Network for Image Synthesis , 2018, BMVC.

[10]  Hayit Greenspan,et al.  Rapid AI Development Cycle for the Coronavirus (COVID-19) Pandemic: Initial Results for Automated Detection & Patient Monitoring using Deep Learning CT Image Analysis , 2020, ArXiv.

[11]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  X. Qi,et al.  Machine learning-based CT radiomics model for predicting hospital stay in patients with pneumonia associated with SARS-CoV-2 infection: A multicenter study , 2020, medRxiv.

[13]  N. Yu,et al.  Quantitative computed tomography analysis for stratifying the severity of Coronavirus Disease 2019 , 2020, Journal of Pharmaceutical Analysis.

[14]  Jeffrey L. Gunter,et al.  Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks , 2018, SASHIMI@MICCAI.

[15]  Antonio M. López,et al.  The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Dinggang Shen,et al.  Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation, and Diagnosis for COVID-19 , 2020, IEEE Reviews in Biomedical Engineering.

[17]  Jing Xu,et al.  MiniSeg: An Extremely Minimum Network for Efficient COVID-19 Segmentation , 2020, AAAI.

[18]  William Parker,et al.  A Weakly Supervised Consistency-based Learning Method for COVID-19 Segmentation in CT Images , 2020, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[19]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Taesung Park,et al.  Semantic Image Synthesis With Spatially-Adaptive Normalization , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[22]  Chao Yang,et al.  A Survey on Deep Transfer Learning , 2018, ICANN.

[23]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[24]  Cornelia Caragea,et al.  Disaster Response Aided by Tweet Classification with a Domain Adaptation Approach , 2018 .

[25]  A. Fenster,et al.  Evaluation of Segmentation algorithms for Medical Imaging , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[26]  Nassir Navab,et al.  Domain and Geometry Agnostic CNNs for Left Atrium Segmentation in 3D Ultrasound , 2018, MICCAI.

[27]  Mengjie Zhang,et al.  Domain Adaptive Neural Networks for Object Recognition , 2014, PRICAI.

[28]  Swami Sankaranarayanan,et al.  Learning from Synthetic Data: Addressing Domain Shift for Semantic Segmentation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[30]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[31]  Daniel Cremers,et al.  Associative Domain Adaptation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[32]  Yue Zhang,et al.  Task Driven Generative Modeling for Unsupervised Domain Adaptation: Application to X-ray Image Segmentation , 2018, MICCAI.

[33]  Qilong Wang,et al.  Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[35]  Mert R. Sabuncu,et al.  Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels , 2018, NeurIPS.

[36]  Hanseok Ko,et al.  COVID-19 CT Image Synthesis with a Conditional Generative Adversarial Network , 2020, IEEE journal of biomedical and health informatics.

[37]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[38]  In So Kweon,et al.  Unsupervised Intra-Domain Adaptation for Semantic Segmentation Through Self-Supervision , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Jean-Philippe Thiran,et al.  Efficient Active Learning for Image Classification and Segmentation using a Sample Selection and Conditional Generative Adversarial Network , 2018, MICCAI.

[40]  Wenyu Liu,et al.  Deep Learning-based Detection for COVID-19 from Chest CT using Weak Label , 2020, medRxiv.

[41]  Philip David,et al.  Domain Adaptation for Semantic Segmentation of Urban Scenes , 2017 .

[42]  Yaozong Gao,et al.  Lung Infection Quantification of COVID-19 in CT Images with Deep Learning , 2020, ArXiv.

[43]  Patrick Pérez,et al.  ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).