Explainable Deep Learning for Covid-19 Detection Using Chest X-ray and CT-Scan Images

Recently, Artificial Intelligence (AI) and more particularly Deep Learning (DL) applications gained significant importance in several domains such as computer vision, robotics, medical imaging, etc. Despite the excellent results of AI models, in terms of precision and performance, their decisions are not always interpretable and explainable, which makes from them a black box. Since May 2018, the general data protection regulation (GDPR) requires a right of explanation for the output of an algorithm, which is necessary and justified for several examples such as autonomous cars and computer-aided diagnosis (CAD) systems. As a result, a high interest in terms of research has been given recently to the domain of Explainable Artificial Intelligence (XAI). In this book chapter, we propose an approach for explaining Deep Learning algorithms when applied to image classification and segmentation. The proposed approach allows to provide the most appropriate explanation method and the most accurate and explainable DL model. As a use case, we applied our approach for explaining DL models used Covid-19 image classification and segmentation with two modalities: X-ray and CT-scan images. Experimental results showed the interest of our explanation approach within three facts: (1) identification of the most interpretable DL model;(2) measurement of positive and negative contributions of input parameters (image pixels) in the decision of DL models;(3) detection of data (training and validation datasets) biases, where the deep neural networks are focusing on image regions that are not supposed to be important. The provided explanations were evaluated by doctors and physicians who confirmed the accuracy of our results. © 2022, Springer Nature Switzerland AG.

[1]  Sidi Ahmed Mahmoudi,et al.  Multi-GPU based event detection and localization using high definition videos , 2014, 2014 International Conference on Multimedia Computing and Systems (ICMCS).

[2]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[3]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Alexander Binder,et al.  Explaining nonlinear classification decisions with deep Taylor decomposition , 2015, Pattern Recognit..

[5]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[6]  Sidi Ahmed Mahmoudi,et al.  Multi-CPU/Multi-GPU Based Framework for Multimedia Processing , 2015, CIIA.

[7]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Daniel S. Kermany,et al.  Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning , 2018, Cell.

[9]  Alexander Binder,et al.  Evaluating the Visualization of What a Deep Neural Network Has Learned , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Klaus-Robert Müller,et al.  Learning how to explain neural networks: PatternNet and PatternAttribution , 2017, ICLR.

[11]  Joseph Paul Cohen,et al.  COVID-19 Image Data Collection , 2020, ArXiv.

[12]  Sidi Ahmed Mahmoudi,et al.  TAKING ADVANTAGE OF HETEROGENEOUS PLATFORMS IN IMAGE AND VIDEO PROCESSING , 2014, HiPC 2014.

[13]  P. Xie,et al.  COVID-CT-Dataset: A CT Scan Dataset about COVID-19 , 2020, ArXiv.

[14]  Sidi Ahmed Mahmoudi,et al.  A new self-adapting architecture for feature detection , 2012, 22nd International Conference on Field Programmable Logic and Applications (FPL).

[15]  Mohammad Rahimzadeh,et al.  A New Modified Deep Convolutional Neural Network for Detecting COVID-19 from X-ray Images , 2020, ArXiv.

[16]  Eneldo Loza Mencía,et al.  DeepRED - Rule Extraction from Deep Neural Networks , 2016, DS.

[17]  Sidi Ahmed Mahmoudi,et al.  A Portable Multi-CPU/Multi-GPU Based Vertebra Localization in Sagittal MR Images , 2014, ICIAR.

[18]  Ankur Taly,et al.  Axiomatic Attribution for Deep Networks , 2017, ICML.

[19]  Been Kim,et al.  Towards A Rigorous Science of Interpretable Machine Learning , 2017, 1702.08608.

[20]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Till Döhmen,et al.  DeepCOVIDExplainer: Explainable COVID-19 Diagnosis from Chest X-ray Images , 2020, 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[22]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Abolfazl Attar,et al.  A modified deep convolutional neural network for detecting COVID-19 and pneumonia from chest X-ray images based on the concatenation of Xception and ResNet50V2 , 2020, Informatics in Medicine Unlocked.

[24]  Avanti Shrikumar,et al.  Learning Important Features Through Propagating Activation Differences , 2017, ICML.

[25]  Piyush Kumar Shukla,et al.  Deep Transfer Learning Based Classification Model for COVID-19 Disease , 2020, IRBM.

[26]  Shaikh Anowarul Fattah,et al.  CovXNet: A multi-dilation convolutional neural network for automatic COVID-19 and other pneumonia detection from chest X-ray images with transferable multi-receptive feature optimization , 2020, Computers in Biology and Medicine.

[27]  Andrew Y. Ng,et al.  CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning , 2017, ArXiv.

[28]  Mamun Bin Ibne Reaz,et al.  Can AI Help in Screening Viral and COVID-19 Pneumonia? , 2020, IEEE Access.

[29]  Till Döhmen,et al.  DeepCOVIDExplainer: Explainable COVID-19 Diagnosis Based on Chest X-ray Images , 2020 .

[30]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Martin Wattenberg,et al.  SmoothGrad: removing noise by adding noise , 2017, ArXiv.

[32]  Sidi Ahmed Mahmoudi,et al.  Multimedia processing using deep learning technologies, high-performance computing cloud resources, and Big Data volumes , 2020, Concurr. Comput. Pract. Exp..

[33]  Sidi Ahmed Mahmoudi,et al.  Cloud-based platform for computer vision applications , 2017, ICSDE.

[34]  Md. Mohaiminul Islam,et al.  COVID-DenseNet: A Deep Learning Architecture to Detect COVID-19 from Chest Radiology Images , 2020 .

[35]  Manjit Kaur,et al.  Automated Deep Transfer Learning-Based Approach for Detection of COVID-19 Infection in Chest X-rays , 2020, IRBM.

[36]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[37]  Klaus-Robert Müller,et al.  Investigating the influence of noise and distractors on the interpretation of neural networks , 2016, ArXiv.

[38]  Chenchen Liu,et al.  How convolutional neural networks see the world - A survey of convolutional neural network visualization methods , 2018, Math. Found. Comput..

[39]  Lalana Kagal,et al.  Explaining Explanations: An Overview of Interpretability of Machine Learning , 2018, 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA).

[40]  Antonella Santone,et al.  Explainable Deep Learning for Pulmonary Disease and Coronavirus COVID-19 Detection from X-rays , 2020, Computer Methods and Programs in Biomedicine.

[41]  Deniz Korkmaz,et al.  COVIDiagnosis-Net: Deep Bayes-SqueezeNet based diagnosis of the coronavirus disease 2019 (COVID-19) from X-ray images , 2020, Medical Hypotheses.

[42]  Klaus-Robert Müller,et al.  iNNvestigate neural networks! , 2018, J. Mach. Learn. Res..

[43]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[44]  Sidi Ahmed Mahmoudi,et al.  Real Time GPU-Based Segmentation and Tracking of the Left Ventricle on 2D Echocardiography , 2016, IWBBIO.

[45]  Kaijin Xu,et al.  A Deep Learning System to Screen Novel Coronavirus Disease 2019 Pneumonia , 2020, Engineering.

[46]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[47]  Sidi Ahmed Mahmoudi,et al.  Real Time Web-based Toolbox for Computer Vision , 2018 .

[48]  Klaus-Robert Müller,et al.  Layer-Wise Relevance Propagation: An Overview , 2019, Explainable AI.

[49]  Sidi Ahmed Mahmoudi,et al.  Towards a smart selection of resources in the cloud for low‐energy multimedia processing , 2017, Concurr. Comput. Pract. Exp..

[50]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[51]  Alexander Binder,et al.  On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation , 2015, PloS one.

[52]  Georg Langs,et al.  Automatic lung segmentation in routine imaging is primarily a data diversity problem, not a methodology problem , 2020, European Radiology Experimental.