Explainable-by-design Semi-Supervised Representation Learning for COVID-19 Diagnosis from CT Imaging

Our motivating application is a real-world problem: COVID-19 classification from CT imaging, for which we present an explainable Deep Learning approach based on a semi-supervised classification pipeline that employs variational autoencoders to extract efficient feature embedding. We have optimized the architecture of two different networks for CT images: (i) a novel conditional variational autoencoder (CVAE) with a specific architecture that integrates the class labels inside the encoder layers and uses side information with shared attention layers for the encoder, which make the most of the contextual clues for representation learning, and (ii) a downstream convolutional neural network for supervised classification using the encoder structure of the CVAE. With the explainable classification results, the proposed diagnosis system is very effective for COVID-19 classification. Based on the promising results obtained qualitatively and quantitatively, we envisage a wide deployment of our developed technique in large-scale clinical studies.Code is available at this https URL.

[1]  Yuedong Yang,et al.  Deep Learning Enables Accurate Diagnosis of Novel Coronavirus (COVID-19) With CT Images , 2021, Ieee/Acm Transactions on Computational Biology and Bioinformatics.

[2]  Wenyu Liu,et al.  A Weakly-Supervised Framework for COVID-19 Classification and Lesion Localization From Chest CT , 2020, IEEE Transactions on Medical Imaging.

[3]  Sebasti'an Amador S'anchez,et al.  Comparative study of deep learning methods for the automatic segmentation of lung, lesion and lesion type in CT scans of COVID-19 patients , 2020, ArXiv.

[4]  Rabha W. Ibrahim,et al.  Classification of Covid-19 Coronavirus, Pneumonia and Healthy Lungs in CT Scans Using Q-Deformed Entropy and Deep Learning Features , 2020, Entropy.

[5]  Ali Gholamrezanezhad,et al.  Coronavirus disease 2019 (COVID-19) imaging reporting and data system (COVID-RADS) and common lexicon: a proposal based on the imaging data of 37 studies , 2020, European Radiology.

[6]  Bram van Ginneken,et al.  CO-RADS – A categorical CT assessment scheme for patients with suspected COVID-19: definition and evaluation , 2020, Radiology.

[7]  X. He,et al.  Sample-Efficient Deep Learning for COVID-19 Diagnosis Based on CT Scans , 2020, medRxiv.

[8]  Ming-Ming Cheng,et al.  JCS: An Explainable COVID-19 Diagnosis System by Joint Classification and Segmentation , 2020, IEEE Transactions on Image Processing.

[9]  Yuan Gao,et al.  Weakly Supervised Deep Learning for COVID-19 Infection Detection and Classification From CT Images , 2020, IEEE Access.

[10]  H. Kauczor,et al.  The Role of Chest Imaging in Patient Management during the COVID-19 Pandemic: A Multinational Consensus Statement from the Fleischner Society , 2020, Radiology.

[11]  Weiliang Tang,et al.  Chest CT for detecting COVID-19: a systematic review and meta-analysis of diagnostic accuracy , 2020, European Radiology.

[12]  A. Amyar,et al.  Multi-task Deep Learning Based CT Imaging Analysis For COVID-19: Classification and Segmentation , 2020, medRxiv.

[13]  P. Xie,et al.  COVID-CT-Dataset: A CT Scan Dataset about COVID-19 , 2020, ArXiv.

[14]  Jonathan H. Chung,et al.  Radiological Society of North America Expert Consensus Statement on Reporting Chest CT Findings Related to COVID-19. Endorsed by the Society of Thoracic Radiology, the American College of Radiology, and RSNA , 2020, Journal of thoracic imaging.

[15]  Haibo Xu,et al.  AI-assisted CT imaging analysis for COVID-19 screening: Building and deploying a medical AI system , 2020, Applied Soft Computing.

[16]  Yaozong Gao,et al.  Large-scale screening to distinguish between COVID-19 and community-acquired pneumonia using infection size-aware classification , 2021, Physics in medicine and biology.

[17]  K. Cao,et al.  Artificial Intelligence Distinguishes COVID-19 from Community Acquired Pneumonia on Chest CT , 2020, Radiology.

[18]  K. Cao,et al.  Using Artificial Intelligence to Detect COVID-19 and Community-acquired Pneumonia Based on Pulmonary CT: Evaluation of the Diagnostic Accuracy , 2020 .

[19]  Hayit Greenspan,et al.  Rapid AI Development Cycle for the Coronavirus (COVID-19) Pandemic: Initial Results for Automated Detection & Patient Monitoring using Deep Learning CT Image Analysis , 2020, ArXiv.

[20]  D. Shen,et al.  Lung Infection Quantification of COVID-19 in CT Images with Deep Learning , 2020, ArXiv.

[21]  Yuedong Yang,et al.  Deep Learning Enables Accurate Diagnosis of Novel Coronavirus (COVID-19) With CT Images , 2020, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[22]  Kaijin Xu,et al.  A Deep Learning System to Screen Novel Coronavirus Disease 2019 Pneumonia , 2020, Engineering.

[23]  Z. Fayad,et al.  Chest CT Findings in Coronavirus Disease-19 (COVID-19): Relationship to Duration of Infection , 2020, Radiology.

[24]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[25]  Z. Fayad,et al.  CT Imaging Features of 2019 Novel Coronavirus (2019-nCoV) , 2020, Radiology.

[26]  Kai Zhao,et al.  Res2Net: A New Multi-Scale Backbone Architecture , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Xavier Binefa,et al.  Learning Disentangled Representations with Reference-Based Variational Autoencoders , 2019, ICLR 2019.

[28]  David Pfau,et al.  Towards a Definition of Disentangled Representations , 2018, ArXiv.

[29]  Sebastian Kmiec,et al.  Learnable Pooling Methods for Video Classification , 2018, ECCV Workshops.

[30]  Nima Tajbakhsh,et al.  UNet++: A Nested U-Net Architecture for Medical Image Segmentation , 2018, DLMIA/ML-CDS@MICCAI.

[31]  Dmitry Vetrov,et al.  Variational Autoencoder with Arbitrary Conditioning , 2018, ICLR.

[32]  Zhe Li,et al.  Evaluate the Malignancy of Pulmonary Nodules Using the 3-D Deep Leaky Noisy-OR Network , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[33]  Hongyi Zhang,et al.  mixup: Beyond Empirical Risk Minimization , 2017, ICLR.

[34]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Tao Mei,et al.  Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine-Grained Image Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Sébastien Ourselin,et al.  Generalised Wasserstein Dice Score for Imbalanced Multi-class Segmentation using Holistic Convolutional Networks , 2017, BrainLes@MICCAI.

[37]  Ankur Taly,et al.  Axiomatic Attribution for Deep Networks , 2017, ICML.

[38]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[40]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[42]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[43]  Anna Shcherbina,et al.  Not Just a Black Box: Learning Important Features Through Propagating Activation Differences , 2016, ArXiv.

[44]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[45]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Honglak Lee,et al.  Learning Structured Output Representation using Deep Conditional Generative Models , 2015, NIPS.

[48]  T. Pajdla,et al.  NetVLAD: CNN Architecture for Weakly Supervised Place Recognition , 2015, Computer Vision and Pattern Recognition.

[49]  Alexander Binder,et al.  On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation , 2015, PloS one.

[50]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[51]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[52]  Max Welling,et al.  Semi-supervised Learning with Deep Generative Models , 2014, NIPS.

[53]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[55]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[56]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[57]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Eugene Demidenko,et al.  Confidence intervals and bands for the binormal ROC curve revisited , 2012, Journal of applied statistics.

[59]  Richard C. Pais,et al.  The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. , 2011, Medical physics.

[60]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[61]  Marek J. Druzdzel,et al.  Learning Bayesian network parameters from small data sets: application of Noisy-OR gates , 2001, Int. J. Approx. Reason..

[62]  Klaus-Robert Müller,et al.  Layer-Wise Relevance Propagation: An Overview , 2019, Explainable AI.

[63]  Peter Filzmoser,et al.  nsROC: An R package for Non-Standard ROC Curve Analysis , 2018, R J..