3D Tomographic Pattern Synthesis for Enhancing the Quantification of COVID-19

The Coronavirus Disease (COVID-19) has affected 1.8 million people and resulted in more than 110,000 deaths as of April 12, 2020. Several studies have shown that tomographic patterns seen on chest Computed Tomography (CT), such as ground-glass opacities, consolidations, and crazy paving pattern, are correlated with the disease severity and progression. CT imaging can thus emerge as an important modality for the management of COVID-19 patients. AI-based solutions can be used to support CT based quantitative reporting and make reading efficient and reproducible if quantitative biomarkers, such as the Percentage of Opacity (PO), can be automatically computed. However, COVID-19 has posed unique challenges to the development of AI, specifically concerning the availability of appropriate image data and annotations at scale. In this paper, we propose to use synthetic datasets to augment an existing COVID-19 database to tackle these challenges. We train a Generative Adversarial Network (GAN) to inpaint COVID-19 related tomographic patterns on chest CTs from patients without infectious diseases. Additionally, we leverage location priors derived from manually labeled COVID-19 chest CTs patients to generate appropriate abnormality distributions. Synthetic data are used to improve both lung segmentation and segmentation of COVID-19 patterns by adding 20% of synthetic data to the real COVID-19 training data. We collected 2143 chest CTs, containing 327 COVID-19 positive cases, acquired from 12 sites across 7 countries. By testing on 100 COVID-19 positive and 100 control cases, we show that synthetic data can help improve both lung segmentation (+6.02% lesion inclusion rate) and abnormality segmentation (+2.78% dice coefficient), leading to an overall more accurate PO computation (+2.82% Pearson coefficient).

[1]  Gillian Macnaught,et al.  Unsupervised learning for cross-domain medical image synthesis using deformation invariant cycle consistency networks , 2018, SASHIMI@MICCAI.

[2]  Nilo Stolte.  Graphics using Implicit Surfaces with Interval Arithmetic based Recursive Voxelization , 2003, Computer Graphics and Imaging.

[3]  Jonathan H. Chung,et al.  Essentials for Radiologists on COVID-19: An Update—Radiology Scientific Expert Panel , 2020, Radiology.

[4]  Daguang Xu,et al.  Automatic Liver Segmentation Using an Adversarial Image-to-Image Network , 2017, MICCAI.

[5]  K. Yuen,et al.  Clinical Characteristics of Coronavirus Disease 2019 in China , 2020, The New England journal of medicine.

[6]  Changhee Han,et al.  Synthesizing Diverse Lung Nodules Wherever Massively: 3D Multi-Conditional GAN-Based CT Image Augmentation for Object Detection , 2019, 2019 International Conference on 3D Vision (3DV).

[7]  Bo Xu,et al.  A deep learning algorithm using CT images to screen for Corona virus disease (COVID-19) , 2020, European Radiology.

[8]  Yicheng Fang,et al.  Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR , 2020, Radiology.

[9]  Su Ruan,et al.  Medical Image Synthesis with Context-Aware Generative Adversarial Networks , 2016, MICCAI.

[10]  Sasa Grbic,et al.  Automated Quantification of CT Patterns Associated with COVID-19 from Chest CT , 2020 .

[11]  Yaozong Gao,et al.  Lung Infection Quantification of COVID-19 in CT Images with Deep Learning , 2020, ArXiv.

[12]  Yuichi Yoshida,et al.  Spectral Normalization for Generative Adversarial Networks , 2018, ICLR.

[13]  S. Inui,et al.  Chest CT Findings in Cases from the Cruise Ship Diamond Princess with Coronavirus Disease (COVID-19) , 2020, Radiology. Cardiothoracic imaging.

[14]  Heinrich Müller,et al.  Improved Laplacian Smoothing of Noisy Surface Meshes , 1999, Comput. Graph. Forum.

[15]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[16]  E. Regan,et al.  Genetic Epidemiology of COPD (COPDGene) Study Design , 2011, COPD.

[17]  D. Lynch,et al.  The National Lung Screening Trial: overview and study design. , 2011, Radiology.

[18]  Z. Fayad,et al.  Chest CT Findings in Coronavirus Disease-19 (COVID-19): Relationship to Duration of Infection , 2020, Radiology.

[19]  Sina Honari,et al.  Distribution Matching Losses Can Hallucinate Features in Medical Image Translation , 2018, MICCAI.

[20]  Frank Hutter,et al.  Decoupled Weight Decay Regularization , 2017, ICLR.

[21]  Ezekiel J Emanuel,et al.  Fair Allocation of Scarce Medical Resources in the Time of Covid-19. , 2020, The New England journal of medicine.

[22]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[23]  Jeffrey L. Gunter,et al.  Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks , 2018, SASHIMI@MICCAI.

[24]  Aaron Carass,et al.  Unpaired Brain MR-to-CT Synthesis Using a Structure-Constrained CycleGAN , 2018, DLMIA/ML-CDS@MICCAI.

[25]  Sotirios A. Tsaftaris,et al.  Adversarial Image Synthesis for Unpaired Multi-modal Cardiac Data , 2017, SASHIMI@MICCAI.

[26]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Ying Zhou,et al.  WGAN-Based Synthetic Minority Over-Sampling Technique: Improving Semantic Fine-Grained Classification for Lung Nodules in CT Images , 2019, IEEE Access.

[28]  Youbao Tang,et al.  CT-Realistic Lung Nodule Simulation from 3D Conditional Generative Adversarial Networks for Robust Lung Segmentation , 2018, MICCAI.

[29]  Jie Yang,et al.  Decompose to manipulate: Manipulable Object Synthesis in 3D Medical Images with Structured Image Decomposition , 2018, ArXiv.

[30]  K. Cao,et al.  Using Artificial Intelligence to Detect COVID-19 and Community-acquired Pneumonia Based on Pulmonary CT: Evaluation of the Diagnostic Accuracy , 2020 .

[31]  Youngjin Yoo,et al.  Quantification of Tomographic Patterns associated with COVID-19 from Chest CT , 2020, ArXiv.

[32]  Xu Sun,et al.  Adaptive Gradient Methods with Dynamic Bound of Learning Rate , 2019, ICLR.

[33]  Jacob D. Furst,et al.  Augmenting LIDC dataset using 3D generative adversarial networks to improve lung nodule detection , 2019, Medical Imaging.

[34]  Jie Yang,et al.  Class-Aware Adversarial Lung Nodule Synthesis In CT Images , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[35]  Dong Yang,et al.  Correlation via synthesis: end-to-end nodule image generation and radiogenomic map learning based on generative adversarial network , 2019, ArXiv.

[36]  G. Chowell,et al.  Estimating the asymptomatic proportion of coronavirus disease 2019 (COVID-19) cases on board the Diamond Princess cruise ship, Yokohama, Japan, 2020 , 2020, Euro surveillance : bulletin Europeen sur les maladies transmissibles = European communicable disease bulletin.

[37]  Q. Tao,et al.  Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases , 2020, Radiology.

[38]  Jun Liu,et al.  Relation Between Chest CT Findings and Clinical Conditions of Coronavirus Disease (COVID-19) Pneumonia: A Multicenter Study. , 2020, AJR. American journal of roentgenology.

[39]  Hayit Greenspan,et al.  GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification , 2018, Neurocomputing.

[40]  Z. Fayad,et al.  CT Imaging Features of 2019 Novel Coronavirus (2019-nCoV) , 2020, Radiology.

[41]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Hayit Greenspan,et al.  Rapid AI Development Cycle for the Coronavirus (COVID-19) Pandemic: Initial Results for Automated Detection & Patient Monitoring using Deep Learning CT Image Analysis , 2020, ArXiv.

[43]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[44]  Yunpeng Ji,et al.  Potential association between COVID-19 mortality and health-care resource availability , 2020, The Lancet Global Health.

[45]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[46]  Daguang Xu,et al.  Tunable CT Lung Nodule Synthesis Conditioned on Background Image and Semantic Features , 2019, SASHIMI@MICCAI.

[47]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[48]  K. Cao,et al.  Artificial Intelligence Distinguishes COVID-19 from Community Acquired Pneumonia on Chest CT , 2020, Radiology.

[49]  H. Kauczor,et al.  The Role of Chest Imaging in Patient Management During the COVID-19 Pandemic , 2020, Chest.

[50]  Dorin Comaniciu,et al.  No Surprises: Training Robust Lung Nodule Detection for Low-Dose CT Scans by Augmenting With Adversarial Attacks , 2021, IEEE Transactions on Medical Imaging.

[51]  Xiaowei Xu,et al.  A Deep Learning System to Screen Novel Coronavirus Disease 2019 Pneumonia , 2020, Engineering.