COVIDScreen: explainable deep learning framework for differential diagnosis of COVID-19 using chest X-rays

COVID-19 has emerged as a global crisis with unprecedented socio-economic challenges, jeopardizing our lives and livelihoods for years to come. The unavailability of vaccines for COVID-19 has rendered rapid testing of the population instrumental in order to contain the exponential rise in cases of infection. Shortage of RT-PCR test kits and delays in obtaining test results calls for alternative methods of rapid and reliable diagnosis. In this article, we propose a novel deep learning-based solution using chest X-rays which can help in rapid triaging of COVID-19 patients. The proposed solution uses image enhancement, image segmentation, and employs a modified stacked ensemble model consisting of four CNN base-learners along with Naive Bayes as meta-learner to classify chest X-rays into three classes viz. COVID-19, pneumonia, and normal. An effective pruning strategy as introduced in the proposed framework results in increased model performance, generalizability, and decreased model complexity. We incorporate explainability in our article by using Grad-CAM visualization in order to establish trust in the medical AI system. Furthermore, we evaluate multiple state-of-the-art GAN architectures and their ability to generate realistic synthetic samples of COVID-19 chest X-rays to deal with limited numbers of training samples. The proposed solution significantly outperforms existing methods, with 98.67% accuracy, 0.98 Kappa score, and F-1 scores of 100, 98, and 98 for COVID-19, normal, and pneumonia classes, respectively, on standard datasets. The proposed solution can be used as one element of patient evaluation along with gold-standard clinical and laboratory testing.

[1]  Mohamed Medhat Gaber,et al.  Classification of COVID-19 in chest X-ray images using DeTraC deep convolutional neural network , 2020, Applied Intelligence.

[2]  Joseph Paul Cohen,et al.  COVID-19 Image Data Collection , 2020, ArXiv.

[3]  Rohan Pandey,et al.  Cervical Cancer Diagnosis using CervixNet - A Deep Learning Approach , 2019, 2019 IEEE 19th International Conference on Bioinformatics and Bioengineering (BIBE).

[4]  C. Metz Basic principles of ROC analysis. , 1978, Seminars in nuclear medicine.

[5]  J. Rocklöv,et al.  The reproductive number of COVID-19 is higher compared to SARS coronavirus , 2020, Journal of travel medicine.

[6]  Yifan Yu,et al.  CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison , 2019, AAAI.

[7]  Loris Nanni,et al.  A critic evaluation of methods for COVID-19 automatic detection from X-ray images , 2020, Information Fusion.

[8]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[9]  Amit Kumar Jaiswal,et al.  Identifying pneumonia in chest X-rays: A deep learning approach , 2019, Measurement.

[10]  U. Rajendra Acharya,et al.  Automated detection of COVID-19 cases using deep neural networks with X-ray images , 2020, Computers in Biology and Medicine.

[11]  Z. Tong,et al.  Potential Presymptomatic Transmission of SARS-CoV-2, Zhejiang Province, China, 2020 , 2020, Emerging infectious diseases.

[12]  Oleksandr Makeyev,et al.  Neural network with ensembles , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[13]  Jong Chul Ye,et al.  Deep Learning COVID-19 Features on CXR Using Limited Training Data Sets , 2020, IEEE Transactions on Medical Imaging.

[14]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  M. Kuo,et al.  Frequency and Distribution of Chest Radiographic Findings in COVID-19 Positive Patients , 2019, Radiology.

[16]  Hayit Greenspan,et al.  Deep learning with non-medical training used for chest pathology identification , 2015, Medical Imaging.

[17]  Mahdiyar Molahasani Majdabadi,et al.  COVID-CXNet: Detecting COVID-19 in frontal chest X-ray images using deep learning , 2022, Multimedia Tools and Applications.

[18]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[19]  Asif Iqbal Khan,et al.  CoroNet: A deep neural network for detection and diagnosis of COVID-19 from chest x-ray images , 2020, Computer Methods and Programs in Biomedicine.

[20]  Deniz Korkmaz,et al.  COVIDiagnosis-Net: Deep Bayes-SqueezeNet based diagnosis of the coronavirus disease 2019 (COVID-19) from X-ray images , 2020, Medical Hypotheses.

[21]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[22]  Yijing Li,et al.  Learning from class-imbalanced data: Review of methods and applications , 2017, Expert Syst. Appl..

[23]  Rajeev Kumar Singh,et al.  DMENet: Diabetic Macular Edema diagnosis using Hierarchical Ensemble of CNNs , 2020, PloS one.

[24]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[25]  Sidong Liu,et al.  Early diagnosis of Alzheimer's disease with deep learning , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[26]  Baozhong Liu,et al.  Overview of image noise reduction based on non-local mean algorithm , 2018 .

[27]  Hao Chen,et al.  VoxResNet: Deep voxelwise residual networks for brain segmentation from 3D MR images , 2017, NeuroImage.

[28]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Jean-Michel Morel,et al.  Non-Local Means Denoising , 2011, Image Process. Line.

[30]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Hannah R. Meredith,et al.  The Incubation Period of Coronavirus Disease 2019 (COVID-19) From Publicly Reported Confirmed Cases: Estimation and Application , 2020, Annals of Internal Medicine.

[32]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Yin Han,et al.  A Familial Cluster of Infection Associated With the 2019 Novel Coronavirus Indicating Possible Person-to-Person Transmission During the Incubation Period , 2020, The Journal of infectious diseases.

[34]  C. E. WHO Coronavirus Disease (COVID-19) Dashboard , 2020 .

[35]  L. Luo,et al.  Modes of contact and risk of transmission in COVID-19 among close contacts , 2020, medRxiv.

[36]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[37]  Bartosz Krawczyk,et al.  Learning from imbalanced data: open challenges and future directions , 2016, Progress in Artificial Intelligence.

[38]  Chih-Fong Tsai Stacked generalisation: a novel solution to bridge the semantic gap for content-based image retrieval , 2003, Online Inf. Rev..

[39]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[40]  Yi Li,et al.  Viral Pneumonia Screening on Chest X-ray Images Using Confidence-Aware Anomaly Detection , 2020 .

[41]  Alexander Wong,et al.  COVID-Net: a tailored deep convolutional neural network design for detection of COVID-19 cases from chest X-ray images , 2020, Scientific reports.

[42]  Andrew Y. Ng,et al.  CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning , 2017, ArXiv.

[43]  Yiu Chung Lau,et al.  Temporal dynamics in viral shedding and transmissibility of COVID-19 , 2020, Nature Medicine.

[44]  Punam Mangtani,et al.  Estimates of the reproduction numbers of Spanish influenza using morbidity data. , 2007, International journal of epidemiology.

[45]  Abdul Hafeez,et al.  COVID-ResNet: A Deep Learning Framework for Screening of COVID19 from Radiographs , 2020, ArXiv.

[46]  M. McHugh Interrater reliability: the kappa statistic , 2012, Biochemia medica.

[47]  Rinshu Dwivedi,et al.  The incubation period of coronavirus disease (COVID‐19): A tremendous public health threat—Forecasting from publicly available case data in India , 2021, Journal of public affairs.

[48]  W. Bieniecki,et al.  Image Preprocessing for Improving OCR Accuracy , 2007, 2007 International Conference on Perspective Technologies and Methods in MEMS Design.

[49]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[50]  Yue-Shi Lee,et al.  Cluster-based under-sampling approaches for imbalanced data distributions , 2009, Expert Syst. Appl..

[51]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[52]  Manjit Kaur,et al.  Automated Deep Transfer Learning-Based Approach for Detection of COVID-19 Infection in Chest X-rays , 2020, IRBM.

[53]  Keith E. Muller,et al.  Contrast-limited adaptive histogram equalization: speed and effectiveness , 1990, [1990] Proceedings of the First Conference on Visualization in Biomedical Computing.

[54]  Hayit Greenspan,et al.  Chest pathology detection using deep learning with non-medical training , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[55]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[56]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[57]  Megan L Ranney,et al.  Critical Supply Shortages - The Need for Ventilators and Personal Protective Equipment during the Covid-19 Pandemic. , 2020, The New England journal of medicine.

[58]  Z. Fayad,et al.  Artificial intelligence–enabled rapid diagnosis of patients with COVID-19 , 2020, Nature Medicine.

[59]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[60]  M. Robb,et al.  Accelerating Development of SARS-CoV-2 Vaccines - The Role for Controlled Human Infection Models. , 2020, The New England journal of medicine.

[61]  Taghi M. Khoshgoftaar,et al.  Survey on deep learning with class imbalance , 2019, J. Big Data.

[62]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[63]  Nicolò Cesa-Bianchi,et al.  Advances in Neural Information Processing Systems 31 , 2018, NIPS 2018.

[64]  Ioannis D. Apostolopoulos,et al.  Covid-19: automatic detection from X-ray images utilizing transfer learning with convolutional neural networks , 2020, Physical and Engineering Sciences in Medicine.

[65]  Stefan Jaeger,et al.  Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. , 2014, Quantitative imaging in medicine and surgery.

[66]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[67]  Mamun Bin Ibne Reaz,et al.  Can AI Help in Screening Viral and COVID-19 Pneumonia? , 2020, IEEE Access.

[68]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[69]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[70]  M. Binnicker Emergence of a Novel Coronavirus Disease (COVID-19) and the Importance of Diagnostic Testing: Why Partnership between Clinical Laboratories, Public Health Agencies, and Industry Is Essential to Control the Outbreak , 2020, Clinical chemistry.

[71]  Victor Alves,et al.  Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images , 2016, IEEE Transactions on Medical Imaging.

[72]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[73]  Carlos Alberto Silva,et al.  Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images. , 2016, IEEE transactions on medical imaging.

[74]  Anne Kimball,et al.  Presymptomatic SARS-CoV-2 Infections and Transmission in a Skilled Nursing Facility , 2020, The New England journal of medicine.

[75]  Antonella Santone,et al.  Explainable Deep Learning for Pulmonary Disease and Coronavirus COVID-19 Detection from X-rays , 2020, Computer Methods and Programs in Biomedicine.

[76]  D. Shen,et al.  Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans , 2016, Scientific Reports.

[77]  J. Alex Stark,et al.  Adaptive image contrast enhancement using generalizations of histogram equalization , 2000, IEEE Trans. Image Process..

[78]  J. Rhee,et al.  Cluster of Coronavirus Disease Associated with Fitness Dance Classes, South Korea , 2020, Emerging infectious diseases.

[79]  M. Usman Akram Retinal Image Preprocessing: Background and Noise Segmentation , 2012 .

[80]  Melina Hosseiny,et al.  Coronavirus (COVID-19) Outbreak: What the Department of Radiology Should Know , 2020, Journal of the American College of Radiology.

[81]  Jonathan H. Chung,et al.  Essentials for Radiologists on COVID-19: An Update—Radiology Scientific Expert Panel , 2020, Radiology.

[82]  Eleftherios Trivizakis,et al.  Interpretable artificial intelligence framework for COVID-19 screening on chest X-rays , 2020, Experimental and therapeutic medicine.

[83]  Tim Miller,et al.  Explanation in Artificial Intelligence: Insights from the Social Sciences , 2017, Artif. Intell..

[84]  H. Commandeur,et al.  Risk in Vaccine Research and Development Quantified , 2013, PloS one.

[85]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[86]  Ali Narin,et al.  Automatic detection of coronavirus disease (COVID-19) using X-ray images and deep convolutional neural networks , 2020, Pattern Analysis and Applications.

[87]  Benoit M. Dawant,et al.  Morphometric analysis of white matter lesions in MR images: method and validation , 1994, IEEE Trans. Medical Imaging.

[88]  Vinay Namboodiri,et al.  CovidAID: COVID-19 Detection Using Chest X-Ray , 2020, ArXiv.

[89]  Nico Karssemeijer,et al.  Unsupervised Deep Learning Applied to Breast Density Segmentation and Mammographic Risk Scoring , 2016, IEEE Transactions on Medical Imaging.