One Shot Model For COVID-19 Classification and Lesions Segmentation In Chest CT Scans Using LSTM With Attention Mechanism

We present a model that fuses instance segmentation, Long Short-Term Memory Network and Attention mechanism to predict COVID-19 and segment chest CT scans. The model works by extracting a sequence of Regions of Interest that contain class-relevant information, and applies two Long Short-Term Memory networks with attention to this sequence to extract class-relevant features. The model is trained in one shot: both segmentation and classification branches, using two different sets of data. We achieve a 95.74% COVID-19 sensitivity, 98.13% Common Pneumonia sensitivity, 99.27% Control sensitivity and 98.15% class-adjusted F1 score on the main dataset of 21191 chest CT scan slices, and also run a number of ablation studies in which we achieve 97.73% COVID-19 sensitivity and 98.41% F1 score. All source code and models are available on https://github.com/AlexTS1980/COVID-LSTM-Attention.

[1]  Yuedong Yang,et al.  Deep Learning Enables Accurate Diagnosis of Novel Coronavirus (COVID-19) With CT Images , 2020, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[2]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[3]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[4]  Dahai Zhao,et al.  A comparative study on the clinical features of COVID-19 pneumonia to other pneumonias , 2020, Clinical infectious diseases : an official publication of the Infectious Diseases Society of America.

[5]  Yun Bian,et al.  Comparison of chest CT findings between COVID-19 pneumonia and other types of viral pneumonia: a two-center retrospective study , 2020, European Radiology.

[6]  Chiranjibi Sitaula,et al.  Attention-based VGG-16 model for COVID-19 chest X-ray image classification , 2020, Applied Intelligence.

[7]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[8]  Luigi Cinque,et al.  A light CNN for detecting COVID-19 from CT scans of the chest , 2020, Pattern Recognition Letters.

[9]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[12]  Xiao Ma,et al.  Enhancing Attention-Based LSTM With Position Context for Aspect-Level Sentiment Classification , 2019, IEEE Access.

[13]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  Jun Liu,et al.  CT Scans of Patients with 2019 Novel Coronavirus (COVID-19) Pneumonia , 2020, Theranostics.

[15]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[16]  Alexander Wong,et al.  COVIDNet-CT: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases From Chest CT Images , 2020, Frontiers in Medicine.

[17]  Hongbing Lu,et al.  Prior-Attention Residual Learning for More Discriminative COVID-19 Screening in CT Images , 2020, IEEE Transactions on Medical Imaging.

[18]  A. Ter-Sarkisov,et al.  One Shot Model For The Prediction of COVID-19 And Lesions Segmentation In Chest CT Scans Through The Affinity Among Lesion Mask Features , 2021, Applied Soft Computing.

[19]  Ming-Ming Cheng,et al.  JCS: An Explainable COVID-19 Diagnosis System by Joint Classification and Segmentation , 2020, IEEE Transactions on Image Processing.

[20]  Zheng Wang,et al.  Open resource of clinical data from patients with pneumonia for the prediction of COVID-19 outcomes via deep learning , 2020, Nature Biomedical Engineering.

[21]  P. Xie,et al.  COVID-CT-Dataset: A CT Scan Dataset about COVID-19 , 2020, ArXiv.

[22]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  David Koff,et al.  COVID-Net CT-2: Enhanced Deep Neural Networks for Detection of COVID-19 From Chest CT Images Through Bigger, More Diverse Learning , 2021, Frontiers in Medicine.

[24]  Aram Ter-Sarkisov.  COVID-CT-Mask-Net: Prediction of COVID-19 from CT Scans Using Regional Features , 2020 .

[25]  A. Ter-Sarkisov,et al.  Detection and Segmentation of Lesion Areas in Chest CT Scans For The Prediction of COVID-19 , 2020, medRxiv.

[26]  Md. Milon Islam,et al.  A combined deep CNN-LSTM network for the detection of novel coronavirus (COVID-19) using X-ray images , 2020, Informatics in Medicine Unlocked.

[27]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[29]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[30]  A. Ter-Sarkisov.  Single-Shot Lightweight Model For The Detection of Lesions And The Prediction of COVID-19 From Chest CT Scans , 2020, medRxiv.

[31]  Milan Sonka,et al.  COVID CT-Net: Predicting Covid-19 From Chest CT Images Using Attentional Convolutional Network , 2020, ArXiv.

[32]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[33]  K. Cao,et al.  Using Artificial Intelligence to Detect COVID-19 and Community-acquired Pneumonia Based on Pulmonary CT: Evaluation of the Diagnostic Accuracy , 2020 .