A Teacher-Student Framework with Fourier Augmentation for COVID-19 Infection Segmentation in CT Images

Automatic segmentation of infected regions in computed tomography (CT) images is necessary for the initial diagnosis of COVID-19. Deep-learning-based methods have the potential to automate this task but require a large amount of data with pixel-level annotations. Training a deep network with annotated lung cancer CT images, which are easier to obtain, can alleviate this problem to some extent. However, this approach may suffer from a reduction in performance when applied to unseen COVID-19 images during the testing phase due to the domain shift. In this paper, we propose a novel unsupervised method for COVID-19 infection segmentation that aims to learn the domain-invariant features from lung cancer and COVID-19 images to improve the generalization ability of the segmentation network for use with COVID-19 CT images. To overcome the intensity shift, our method first transforms annotated lung cancer data into the style of unlabeled COVID-19 data using an effective augmentation approach via a Fourier transform. Furthermore, to reduce the distribution shift, we design a teacher-student network to learn rotation-invariant features for segmentation. Experiments demonstrate that even without getting access to the annotations of COVID-19 CT during training, the proposed network can achieve a state-of-the-art segmentation performance on COVID-19 images.

[1]  장윤희,et al.  Y. , 2003, Industrial and Labor Relations Terms.

[2]  C. Tempany,et al.  Domain adaptation for segmentation of critical structures for prostate cancer therapy , 2021, Scientific Reports.

[3]  Adel OULEFKI,et al.  Automatic COVID-19 lung infected region segmentation and measurement using CT-scans images , 2020, Pattern Recognition.

[4]  Nuo Tong,et al.  Shape constrained fully convolutional DenseNet with adversarial training for multiorgan segmentation on head and neck CT and low-field MR images. , 2019, Medical physics.

[5]  Ronald M. Summers,et al.  Self-learning to detect and segment cysts in lung CT images without manual annotation , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[6]  Hanseok Ko,et al.  Domain Adaptation based COVID-19 CT Lung Infections Segmentation Network , 2020 .

[7]  Hui Shen,et al.  Dual-branch combination network (DCN): Towards accurate diagnosis and lesion segmentation of COVID-19 using CT images , 2020, Medical Image Analysis.

[8]  Stefano Soatto,et al.  FDA: Fourier Domain Adaptation for Semantic Segmentation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Kup-Sze Choi,et al.  Local and Global Structure-Aware Entropy Regularized Mean Teacher Model for 3D Left Atrium Segmentation , 2020, MICCAI.

[10]  Bo Du,et al.  Self-Ensembling Attention Networks: Addressing Domain Shift for Semantic Segmentation , 2019, AAAI.

[11]  A. Fenster,et al.  Evaluation of Segmentation algorithms for Medical Imaging , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[12]  Yike Guo,et al.  Transfer Learning from Partial Annotations for Whole Brain Segmentation , 2019, DART/MIL3ID@MICCAI.

[13]  Rich Caruana,et al.  Model compression , 2006, KDD '06.

[14]  P. Horby,et al.  A novel coronavirus outbreak of global health concern , 2020, The Lancet.

[15]  Few-shot Learning for CT Scan based COVID-19 Diagnosis , 2021, ArXiv.

[16]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[17]  Chi-Wing Fu,et al.  Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation , 2019, MICCAI.

[18]  P. Alam,et al.  H , 1887, High Explosives, Propellants, Pyrotechnics.

[19]  Mitko Veta,et al.  Adversarial Training and Dilated Convolutions for Brain MRI Segmentation , 2017, DLMIA/ML-CDS@MICCAI.

[20]  Richard C. Pais,et al.  The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. , 2011, Medical physics.

[21]  Yu Zhang,et al.  Momentum contrastive learning for few-shot COVID-19 diagnosis from chest CT images , 2020, Pattern Recognition.

[22]  William Parker,et al.  A Weakly Supervised Consistency-based Learning Method for COVID-19 Segmentation in CT Images , 2020, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[23]  Yaozong Gao,et al.  Dual-Sampling Attention Network for Diagnosis of COVID-19 From Community Acquired Pneumonia , 2020, IEEE Transactions on Medical Imaging.

[24]  Changick Kim,et al.  Self-Ensembling With GAN-Based Data Augmentation for Domain Adaptation in Semantic Segmentation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[25]  Miss A.O. Penney.  (b) , 1974, The New Yale Book of Quotations.

[26]  Li Tong,et al.  Advancing Medical Imaging Informatics by Deep Learning-Based Domain Adaptation , 2020, Yearbook of Medical Informatics.

[27]  Yiming Li,et al.  Semi-Supervised Brain Lesion Segmentation with an Adapted Mean Teacher Model , 2019, IPMI.

[28]  Joseph O. Deasy,et al.  Tumor-Aware, Adversarial Domain Adaptation from CT to MRI for Lung Cancer Segmentation , 2018, MICCAI.

[29]  Yu Shi,et al.  Robust Pancreatic Ductal Adenocarcinoma Segmentation with Multi-Institutional Multi-Phase Partially-Annotated CT Scans , 2020, MICCAI.

[30]  Dong Liu,et al.  COVID-19 and early-stage lung cancer both featuring ground-glass opacities: a propensity score-matched study , 2020, Translational lung cancer research.

[31]  K. Cao,et al.  Artificial Intelligence Distinguishes COVID-19 from Community Acquired Pneumonia on Chest CT , 2020, Radiology.

[32]  Su Ruan,et al.  An automatic COVID-19 CT segmentation based on U-Net with attention mechanism , 2020 .

[33]  Jianjiang Feng,et al.  Development and evaluation of an artificial intelligence system for COVID-19 diagnosis , 2020, Nature Communications.

[34]  Julia Adler-Milstein,et al.  Sharing clinical data electronically: a critical challenge for fixing the health care system. , 2012, JAMA.

[35]  Q. Tao,et al.  Serial Quantitative Chest CT Assessment of COVID-19: A Deep Learning Approach , 2020, Radiology. Cardiothoracic imaging.

[36]  Ling Shao,et al.  Inf-Net: Automatic COVID-19 Lung Infection Segmentation From CT Images , 2020, IEEE Transactions on Medical Imaging.

[37]  P. Alam.  ‘E’ , 2021, Composites Engineering: An A–Z Guide.

[38]  Andrew Gordon Wilson,et al.  There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average , 2018, ICLR.

[39]  ShanmugaSundari N,et al.  Automatic COVID-19 Lung Infection Segmentation from CT Images , 2021, SSRN Electronic Journal.

[40]  Dinggang Shen,et al.  Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation, and Diagnosis for COVID-19 , 2020, IEEE Reviews in Biomedical Engineering.

[41]  Richard J. Chen,et al.  Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images , 2018, IEEE Transactions on Medical Imaging.

[42]  Jianjiang Feng,et al.  Longitudinal Assessment of COVID-19 Using a Deep Learning–based Quantitative CT Pipeline: Illustration of Two Cases , 2020, Radiology. Cardiothoracic imaging.

[43]  Zhiyong Xu,et al.  A Noise-Robust Framework for Automatic Segmentation of COVID-19 Pneumonia Lesions From CT Images , 2020, IEEE Transactions on Medical Imaging.

[44]  Farah E. Shamout,et al.  Preserving Patient Privacy while Training a Predictive Model of In-hospital Mortality , 2019, ArXiv.

[45]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[46]  Linga Reddy Cenkeramaddi,et al.  Anam-Net: Anamorphic Depth Embedding-Based Lightweight CNN for Segmentation of Anomalies in COVID-19 Chest CT Images , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[47]  Patrick Pérez,et al.  ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[49]  Xiaolong Qi,et al.  CT Imaging of the 2019 Novel Coronavirus (2019-nCoV) Pneumonia , 2020, Radiology.