Cross-Site Severity Assessment of COVID-19 From CT Images via Domain Adaptation

Early and accurate severity assessment of Coronavirus disease 2019 (COVID-19) based on computed tomography (CT) images offers a great help to the estimation of intensive care unit event and the clinical decision of treatment planning. To augment the labeled data and improve the generalization ability of the classification model, it is necessary to aggregate data from multiple sites. This task faces several challenges including class imbalance between mild and severe infections, domain distribution discrepancy between sites, and presence of heterogeneous features. In this paper, we propose a novel domain adaptation (DA) method with two components to address these problems. The first component is a stochastic class-balanced boosting sampling strategy that overcomes the imbalanced learning problem and improves the classification performance on poorly-predicted classes. The second component is a representation learning that guarantees three properties: 1) domain-transferability by prototype triplet loss, 2) discriminant by conditional maximum mean discrepancy loss, and 3) completeness by multi-view reconstruction loss. Particularly, we propose a domain translator and align the heterogeneous data to the estimated class prototypes (i.e., class centers) in a hyper-sphere manifold. Experiments on cross-site severity assessment of COVID-19 from CT images show that the proposed method can effectively tackle the imbalanced learning problem and outperform recent DA approaches.

[1]  Milan Sonka,et al.  COVID CT-Net: Predicting Covid-19 From Chest CT Images Using Attentional Convolutional Network , 2020, ArXiv.

[2]  D. Shen,et al.  Joint prediction and time estimation of COVID-19 developing severe symptoms using chest CT scan , 2020, Medical Image Analysis.

[3]  Hui Han,et al.  Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning , 2005, ICIC.

[4]  Zhao Wang,et al.  Contrastive Cross-Site Learning With Redesigned Net for COVID-19 CT Classification , 2020, IEEE Journal of Biomedical and Health Informatics.

[5]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[6]  Inderjit S. Dhillon,et al.  Memory Efficient Kernel Approximation , 2014, ICML.

[7]  Jing Zhao,et al.  Early Transmission Dynamics in Wuhan, China, of Novel Coronavirus–Infected Pneumonia , 2020, The New England journal of medicine.

[8]  P. Horby,et al.  A novel coronavirus outbreak of global health concern , 2020, The Lancet.

[9]  Chuan-Xian Ren,et al.  Unsupervised Domain Adaptation via Discriminative Manifold Propagation , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Trevor Darrell,et al.  Semi-Supervised Domain Adaptation via Minimax Entropy , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[11]  Milan Sonka,et al.  COVID TV-UNet: Segmenting COVID-19 Chest CT Images Using Connectivity Imposed U-Net , 2020, ArXiv.

[12]  Zhengming Ding,et al.  Towards Fair Cross-Domain Adaptation via Generative Learning , 2020, ArXiv.

[13]  Yaozong Gao,et al.  Large-scale screening to distinguish between COVID-19 and community-acquired pneumonia using infection size-aware classification , 2021, Physics in medicine and biology.

[14]  Marcus Rohrbach,et al.  Decoupling Representation and Classifier for Long-Tailed Recognition , 2020, ICLR.

[15]  Yu-Chiang Frank Wang,et al.  A Closer Look at Few-shot Classification , 2019, ICLR.

[16]  Yunchao Wei,et al.  Contrastive Adaptation Network for Single- and Multi-Source Domain Adaptation , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Bo Wang,et al.  Moment Matching for Multi-Source Domain Adaptation , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[18]  Dinggang Shen,et al.  Severity assessment of COVID-19 using CT image features and laboratory indices , 2020, Physics in medicine and biology.

[19]  Ioannis D. Apostolopoulos,et al.  Covid-19: automatic detection from X-ray images utilizing transfer learning with convolutional neural networks , 2020, Physical and Engineering Sciences in Medicine.

[20]  C. Baker.  Joint measures and cross-covariance operators , 1973 .

[21]  Xiu-Shen Wei,et al.  BBN: Bilateral-Branch Network With Cumulative Learning for Long-Tailed Visual Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[23]  Yann LeCun,et al.  Dimensionality Reduction by Learning an Invariant Mapping , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[24]  Nour Eldeen M. Khalifa,et al.  Detection of Coronavirus (COVID-19) Associated Pneumonia based on Generative Adversarial Networks and a Fine-Tuned Deep Transfer Learning Model using Chest X-ray Dataset , 2020, ArXiv.

[25]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[26]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Paul L. Rosin,et al.  Self-Paced Balance Learning for Clinical Skin Disease Recognition , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Y. Hu,et al.  Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China , 2020, The Lancet.

[29]  Zhenyue Zhang,et al.  Semi-Supervised Domain Adaptation by Covariance Matching , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Ting Yu,et al.  Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study , 2020, The Lancet.

[31]  Marcus A. Badgeley,et al.  Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study , 2018, PLoS medicine.

[32]  Shuicheng Yan,et al.  Learning Target-Domain-Specific Classifier for Partial Domain Adaptation , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[33]  K. Yuen,et al.  Clinical Characteristics of Coronavirus Disease 2019 in China , 2020, The New England journal of medicine.

[34]  C. Whittaker,et al.  Estimates of the severity of coronavirus disease 2019: a model-based analysis , 2020, The Lancet Infectious Diseases.

[35]  Dean C. Barratt,et al.  Inter-site Variability in Prostate Segmentation Accuracy Using Deep Learning , 2018, MICCAI.

[36]  P. Singh,et al.  Automated assessment of COVID-19 pulmonary disease severity on chest radiographs using convolutional Siamese neural networks , 2020, medRxiv.

[37]  A. P. Gonchar,et al.  MosMedData: data set of 1110 chest CT scans performed during the COVID-19 epidemic , 2020 .

[38]  Michael I. Jordan,et al.  Conditional Adversarial Domain Adaptation , 2017, NeurIPS.

[39]  Youngbin Shin,et al.  COVID-19 Pneumonia Diagnosis Using a Simple 2D Deep Learning Framework With a Single Chest CT Image: Model Development and Validation , 2020, Journal of medical Internet research.

[40]  Gustavo E. A. P. A. Batista,et al.  A study of the behavior of several methods for balancing machine learning training data , 2004, SKDD.

[41]  Dorin Comaniciu,et al.  Automated Quantification of CT Patterns Associated with COVID-19 from Chest CT , 2020, Radiology. Artificial intelligence.

[42]  Yaozong Gao,et al.  Dual-Sampling Attention Network for Diagnosis of COVID-19 From Community Acquired Pneumonia , 2020, IEEE Transactions on Medical Imaging.

[43]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[44]  Hung-Yu Tseng,et al.  Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation , 2020, ICLR.

[45]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[46]  Shuicheng Yan,et al.  Heterogeneous Domain Adaptation via Covariance Structured Feature Translators , 2019, IEEE Transactions on Cybernetics.

[47]  Yuedong Yang,et al.  Deep Learning Enables Accurate Diagnosis of Novel Coronavirus (COVID-19) With CT Images , 2020, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[48]  Xiaowei Xu,et al.  Weakly-Supervised Cross-Domain Adaptation for Endoscopic Lesions Segmentation , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[49]  Dijia Wu,et al.  Diagnosis of Coronavirus Disease 2019 (COVID-19) With Structured Latent Multi-View Representation Learning , 2020, IEEE Transactions on Medical Imaging.

[50]  Haibo He,et al.  ADASYN: Adaptive synthetic sampling approach for imbalanced learning , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[51]  Nikos Paragios,et al.  AI-driven quantification, staging and outcome prediction of COVID-19 pneumonia , 2020, Medical Image Analysis.

[52]  Huazhu Fu,et al.  CPM-Nets: Cross Partial Multi-View Networks , 2019, NeurIPS.

[53]  Lequan Yu,et al.  MS-Net: Multi-Site Network for Improving Prostate Segmentation With Heterogeneous MRI Data , 2020, IEEE Transactions on Medical Imaging.

[54]  Hong Yan,et al.  Learning Kernel for Conditional Moment-Matching Discrepancy-Based Image Classification , 2019, IEEE Transactions on Cybernetics.

[55]  Kayhan Zrar Ghafoor,et al.  Diagnosing COVID-19 pneumonia from x-ray and CT images using deep learning and transfer learning algorithms , 2020, Defense + Commercial Sensing.

[56]  Liang Lin,et al.  Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[57]  D. Shen,et al.  Abnormal lung quantification in chest CT images of COVID‐19 patients with deep learning and its application to severity prediction , 2020, Medical physics.

[58]  Chuan-Xian Ren,et al.  Enhanced Transport Distance for Unsupervised Domain Adaptation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).