Dual-Consistency Semi-Supervised Learning with Uncertainty Quantification for COVID-19 Lesion Segmentation from CT Images

The novel coronavirus disease 2019 (COVID-19) characterized by atypical pneumonia has caused millions of deaths worldwide. Automatically segmenting lesions from chest Computed Tomography (CT) is a promising way to assist doctors in COVID-19 screening, treatment planning, and follow-up monitoring. However, voxel-wise annotations are extremely expert-demanding and scarce, especially when it comes to novel diseases, while an abundance of unlabeled data could be available. To tackle the challenge of limited annotations, in this paper, we propose an uncertainty-guided dual-consistency learning network (UDC-Net) for semi-supervised COVID-19 lesion segmentation from CT images. Specifically, we present a dual-consistency learning scheme that simultaneously imposes image transformation equivalence and feature perturbation invariance to effectively harness the knowledge from unlabeled data. We then quantify the segmentation uncertainty in two forms and employ them together to guide the consistency regularization for more reliable unsupervised learning. Extensive experiments showed that our proposed UDC-Net improves the fully supervised method by 6.3% in Dice and outperforms other competitive semi-supervised approaches by significant margins, demonstrating high potential in real-world clinical practice. (Code is available at https://github.com/poiuohke/UDC-Net ). © 2021, Springer Nature Switzerland AG.

[1]  Samy Bengio,et al.  Understanding deep learning requires rethinking generalization , 2016, ICLR.

[2]  Chao Di,et al.  U1 snRNP regulates cancer cell migration and invasion in vitro , 2020, Nature Communications.

[3]  Yicheng Fang,et al.  Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR , 2020, Radiology.

[4]  Jong Chul Ye,et al.  Deep Learning COVID-19 Features on CXR Using Limited Training Data Sets , 2020, IEEE Transactions on Medical Imaging.

[5]  Zhiyong Xu,et al.  A Noise-Robust Framework for Automatic Segmentation of COVID-19 Pneumonia Lesions From CT Images , 2020, IEEE Transactions on Medical Imaging.

[6]  Marta Luksza,et al.  Artificial intelligence-enabled rapid diagnosis of COVID-19 patients. , 2020, medRxiv : the preprint server for health sciences.

[7]  Lequan Yu,et al.  Deep Mining External Imperfect Data for Chest X-Ray Disease Screening , 2020, IEEE Transactions on Medical Imaging.

[8]  Chi-Wing Fu,et al.  Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation , 2019, MICCAI.

[9]  Bo Wang,et al.  Deep Co-Training for Semi-Supervised Image Recognition , 2018, ECCV.

[10]  Stephan J. Garbin,et al.  Harmonic Networks: Deep Translation and Rotation Equivariance , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Shin Ishii,et al.  Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Hao Chen,et al.  UD-MIL: Uncertainty-Driven Deep Multiple Instance Learning for OCT Image Classification , 2020, IEEE Journal of Biomedical and Health Informatics.

[13]  Wenyu Liu,et al.  A Weakly-Supervised Framework for COVID-19 Classification and Lesion Localization From Chest CT , 2020, IEEE Transactions on Medical Imaging.

[14]  Y. Hu,et al.  Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China , 2020, The Lancet.

[15]  Jianjiang Feng,et al.  Development and evaluation of an artificial intelligence system for COVID-19 diagnosis , 2020, Nature Communications.

[16]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[17]  Xiaojin Zhu,et al.  Semi-Supervised Learning , 2010, Encyclopedia of Machine Learning.

[18]  Hao Chen,et al.  Deep virtual adversarial self-training with consistency regularization for semi-supervised medical image classification , 2021, Medical Image Anal..

[19]  Harri Valpola,et al.  Weight-averaged consistency targets improve semi-supervised deep learning results , 2017, ArXiv.

[20]  Jianhua Yao,et al.  Early triage of critically ill COVID-19 patients using deep learning , 2020, Nature Communications.

[21]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[22]  Rynson W. H. Lau,et al.  Dual Student: Breaking the Limits of the Teacher in Semi-Supervised Learning , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[23]  David Berthelot,et al.  MixMatch: A Holistic Approach to Semi-Supervised Learning , 2019, NeurIPS.

[24]  G. Gao,et al.  A Novel Coronavirus from Patients with Pneumonia in China, 2019 , 2020, The New England journal of medicine.

[25]  C. Hudelot,et al.  Semi-Supervised Semantic Segmentation With Cross-Consistency Training , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  M. Mildner,et al.  Re-epithelialization and immune cell behaviour in an ex vivo human skin model , 2020, Scientific Reports.

[27]  Ling Shao,et al.  Inf-Net: Automatic COVID-19 Lung Infection Segmentation From CT Images , 2020, IEEE Transactions on Medical Imaging.

[28]  Lequan Yu,et al.  Semi-Supervised Medical Image Classification With Relation-Driven Self-Ensembling Model , 2020, IEEE Transactions on Medical Imaging.

[29]  Dong Yang,et al.  Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation , 2020, Medical Image Anal..

[30]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[31]  Sae-Young Chung,et al.  Robust Training with Ensemble Consensus , 2019, ICLR.

[32]  Yuan Zhang,et al.  FocalMix: Semi-Supervised Learning for 3D Medical Image Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Q. Tao,et al.  Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases , 2020, Radiology.

[34]  Holger H. Hoos,et al.  A survey on semi-supervised learning , 2019, Machine Learning.

[35]  Hao Chen,et al.  Towards multi-center glaucoma OCT image screening with semi-supervised joint structure and function multi-task learning , 2020, Medical Image Anal..

[36]  Jun Ma,et al.  Active contour regularized semi-supervised learning for COVID-19 CT infection segmentation with limited annotations , 2020, Physics in medicine and biology.

[37]  Alexander Wong,et al.  COVID-Net: a tailored deep convolutional neural network design for detection of COVID-19 cases from chest X-ray images , 2020, Scientific reports.

[38]  Z. Fayad,et al.  Artificial intelligence–enabled rapid diagnosis of patients with COVID-19 , 2020, Nature Medicine.

[39]  Yaozong Gao,et al.  Hypergraph learning for identification of COVID-19 with CT imaging , 2020, Medical Image Analysis.

[40]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.