COVID-19 identification from volumetric chest CT scans using a progressively resized 3D-CNN incorporating segmentation, augmentation, and class-rebalancing

The novel COVID-19 is a global pandemic disease overgrowing worldwide. Computer-aided screening tools with greater sensitivity are imperative for disease diagnosis and prognosis as early as possible. It also can be a helpful tool in triage for testing and clinical supervision of COVID-19 patients. However, designing such an automated tool from non-invasive radiographic images is challenging as many manually annotated datasets are not publicly available yet, which is the essential core requirement of supervised learning schemes. This article proposes a 3D Convolutional Neural Network (CNN)-based classification approach considering both the inter-and intra-slice spatial voxel information. The proposed system is trained end-to-end on the 3D patches from the whole volumetric Computed Tomography (CT) images to enlarge the number of training samples, performing the ablation studies on patch size determination. We integrate progressive resizing, segmentation, augmentations, and class-rebalancing into our 3D network. The segmentation is a critical prerequisite step for COVID-19 diagnosis enabling the classifier to learn prominent lung features while excluding the outer lung regions of the CT scans. We evaluate all the extensive experiments on a publicly available dataset named MosMed, having binary- and multi-class chest CT image partitions. Our experimental results are very encouraging, yielding areas under the Receiver Operating Characteristics (ROC) curve of 0 . 914 ± 0 . 049 and 0 . 893 ± 0 . 035 for the binary- and multi-class tasks, respectively, applying 5-fold cross-validations. Our method’s promising results delegate it as a favorable aiding tool for clinical practitioners and radiologists to assess COVID-19.

[1]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[2]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[3]  S. P. Morozov,et al.  MosMedData: Chest CT Scans with COVID-19 Related Findings , 2020, medRxiv.

[4]  Mohamed Medhat Gaber,et al.  Classification of COVID-19 in chest X-ray images using DeTraC deep convolutional neural network , 2020, Applied Intelligence.

[5]  U. Rajendra Acharya,et al.  Automated detection of COVID-19 cases using deep neural networks with X-ray images , 2020, Computers in Biology and Medicine.

[6]  P. Angelov,et al.  EXPLAINABLE-BY-DESIGN APPROACH FOR COVID-19 CLASSIFICATION VIA CT-SCAN , 2020, medRxiv.

[7]  R. Jeraj,et al.  Performance and Robustness of Machine Learning-based Radiomic COVID-19 Severity Prediction , 2020, medRxiv.

[8]  P. Khong,et al.  COVID-19 pneumonia: what has CT taught us? , 2020, The Lancet Infectious Diseases.

[9]  Eklas Hossain,et al.  Diabetes Prediction Using Ensembling of Different Machine Learning Classifiers , 2020, IEEE Access.

[10]  Shadi Albarqouni,et al.  Capsule Networks against Medical Imaging Data Challenges , 2018, CVII-STENT/LABELS@MICCAI.

[11]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Ioannis D. Apostolopoulos,et al.  Extracting Possibly Representative COVID-19 Biomarkers from X-ray Images with Deep Learning Approach and Image Data Related to Pulmonary Diseases , 2020, Journal of Medical and Biological Engineering.

[13]  Wenyu Liu,et al.  A Weakly-Supervised Framework for COVID-19 Classification and Lesion Localization From Chest CT , 2020, IEEE Transactions on Medical Imaging.

[14]  Yandre M. G. Costa,et al.  COVID-19 identification in chest X-ray images on flat and hierarchical classification scenarios , 2020, Computer Methods and Programs in Biomedicine.

[15]  U. Rajendra Acharya,et al.  Automated segmentation of exudates, haemorrhages, microaneurysms using single convolutional neural network , 2017, Inf. Sci..

[16]  Md. Kamrul Hasan,et al.  Automatic Mass Detection in Breast Using Deep Convolutional Neural Network and SVM Classifier , 2019, ArXiv.

[17]  Xiangyu Zhang,et al.  ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[19]  Youngbin Shin,et al.  COVID-19 Pneumonia Diagnosis Using a Simple 2D Deep Learning Framework With a Single Chest CT Image: Model Development and Validation , 2020, Journal of medical Internet research.

[20]  Piyush Kumar Shukla,et al.  Deep Transfer Learning Based Classification Model for COVID-19 Disease , 2020, IRBM.

[21]  Concetto Spampinato,et al.  Information Bottleneck Attribution for Visual Explanations of Diagnosis and Prognosis , 2021, MLMI@MICCAI.

[22]  Yi-Chi Wu,et al.  The outbreak of COVID-19: An overview , 2020, Journal of the Chinese Medical Association : JCMA.

[23]  Yung-Chih Wang,et al.  Point-of-Care RNA-Based Diagnostic Device for COVID-19 , 2020, Diagnostics.

[24]  Mohammed Meknassi,et al.  CVDNet: A novel deep learning architecture for detection of coronavirus (Covid-19) from chest x-ray images , 2020, Chaos, Solitons & Fractals.

[25]  Tianqi Chen,et al.  Empirical Evaluation of Rectified Activations in Convolutional Network , 2015, ArXiv.

[26]  Masoumeh Haghpanahi,et al.  Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network , 2019, Nature Medicine.

[27]  Bhuvan Mittal,et al.  CoviNet: Covid-19 diagnosis using machine learning analyses for computerized tomography images , 2021, International Conference on Digital Image Processing.

[28]  Prabira Kumar Sethy,et al.  Detection of Coronavirus Disease (COVID-19) Based on Deep Features , 2020 .

[29]  Jia Deng,et al.  A large-scale hierarchical image database , 2009, CVPR 2009.

[30]  U. Rajendra Acharya,et al.  A deep convolutional neural network model to classify heartbeats , 2017, Comput. Biol. Medicine.

[31]  Juan Manuel Górriz,et al.  Covid-19 classification by FGCNet with deep feature fusion from graph convolutional network and convolutional neural network , 2020, Information Fusion.

[32]  Govardhan Jain,et al.  A deep learning approach to detect Covid-19 coronavirus with X-Ray images , 2020, Biocybernetics and Biomedical Engineering.

[33]  ImageNet Classification with Deep Convolutional Neural , 2013 .

[34]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[35]  Xiangjian He,et al.  Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges , 2019, Journal of Digital Imaging.

[36]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[37]  Taghi M. Khoshgoftaar,et al.  A survey on Image Data Augmentation for Deep Learning , 2019, Journal of Big Data.

[38]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Mohamed Medhat Gaber,et al.  DeTrac: Transfer Learning of Class Decomposed Medical Images in Convolutional Neural Networks , 2020, IEEE Access.

[40]  Aurélien Géron,et al.  Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems , 2017 .

[41]  Y. Chu,et al.  Supervised framework for COVID-19 classification and lesion localization from chest CT , 2020 .

[42]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[43]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Shivajirao M. Jadhav,et al.  Deep convolutional neural network based medical image classification for disease diagnosis , 2019, Journal of Big Data.

[45]  Pierre Soille,et al.  Morphological segmentation of binary patterns , 2009, Pattern Recognit. Lett..

[46]  Yan Zhao,et al.  Clinical Characteristics of 138 Hospitalized Patients With 2019 Novel Coronavirus-Infected Pneumonia in Wuhan, China. , 2020, JAMA.

[47]  Vaishali,et al.  Classification of COVID-19 patients from chest CT images using multi-objective differential evolution–based convolutional neural networks , 2020, European Journal of Clinical Microbiology & Infectious Diseases.

[48]  R. Lu,et al.  Detection of SARS-CoV-2 in Different Types of Clinical Specimens. , 2020, JAMA.

[49]  Ali Narin,et al.  Automatic detection of coronavirus disease (COVID-19) using X-ray images and deep convolutional neural networks , 2020, Pattern Analysis and Applications.

[50]  Dilbag Singh,et al.  Classification of the COVID-19 infected patients using DenseNet201 based deep transfer learning , 2020, Journal of biomolecular structure & dynamics.

[51]  A. Conci,et al.  Hyperparameter Optimization for COVID-19 Pneumonia Diagnosis Based on Chest CT , 2021, Sensors.

[52]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[53]  M. K. Hasan,et al.  Challenges of Deep Learning Methods for COVID-19 Detection Using Public Datasets , 2020, medRxiv.

[54]  Navalgund Rao,et al.  Deep 3D convolution neural network for CT brain hemorrhage classification , 2018, Medical Imaging.

[55]  Di Zhao,et al.  A review of the application of deep learning in medical image classification and segmentation , 2020, Annals of translational medicine.

[56]  Patel Dhruv,et al.  Image Classification Using Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN): A Review , 2020 .

[57]  Manoranjan Paul,et al.  COVID-19 Detection Through Transfer Learning Using Multimodal Imaging Data , 2020, IEEE Access.

[58]  Abnormality detection and intelligent severity assessment of human chest computed tomography scans using deep learning: a case study on SARS-COV-2 assessment , 2021, Journal of ambient intelligence and humanized computing.

[59]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[60]  Christl A. Donnelly,et al.  The impact of COVID-19 and strategies for mitigation and suppression in low- and middle-income countries , 2020, Science.

[61]  Aishwariya Dutta,et al.  Skin Lesion Classification Using Convolutional Neural Network for Melanoma Recognition , 2021, Algorithms for Intelligent Systems.

[62]  Md. Kamrul Hasan,et al.  CVR-Net: A deep convolutional neural network for coronavirus recognition from chest radiography images , 2020, ArXiv.

[63]  M. Ghalambaz,et al.  Using deep learning to learn physics of conduction heat transfer , 2020, Journal of Thermal Analysis and Calorimetry.

[64]  M. K. Hasan,et al.  Skin Lesion Classification Using Convolutional Neural Network for Melanoma Recognition , 2020, medRxiv.

[65]  Fabio A. González,et al.  Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks , 2014, Medical Imaging.

[66]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[67]  Ashutosh Vyas,et al.  Deep Learning for Natural Language Processing , 2016 .

[68]  Ram Bilas Pachori,et al.  Application of deep learning techniques for detection of COVID-19 cases using chest X-ray images: A comprehensive study , 2020, Biomedical Signal Processing and Control.

[69]  Su-Lin Lee,et al.  Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis , 2017, Lecture Notes in Computer Science.

[70]  Mohiuddin Ahmad,et al.  Prediction of Epileptic Seizure by Analysing Time Series EEG Signal Using k-NN Classifier , 2017, Applied bionics and biomechanics.

[71]  U. Rajendra Acharya,et al.  Automated invasive ductal carcinoma detection based using deep transfer learning with whole-slide images , 2020, Pattern Recognit. Lett..

[72]  Thomas Martinetz,et al.  Explainable COVID-19 Detection Using Chest CT Scans and Deep Learning , 2021, Sensors.

[73]  Ting Yu,et al.  Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study , 2020, The Lancet.

[74]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[75]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[76]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[77]  U. Rajendra Acharya,et al.  Arrhythmia detection using deep convolutional neural network with long duration ECG signals , 2018, Comput. Biol. Medicine.

[78]  Javad Alirezaie,et al.  Automatic lung segmentation in CT images using watershed transform , 2005, IEEE International Conference on Image Processing 2005.

[79]  Vijay Vasudevan,et al.  Learning Transferable Architectures for Scalable Image Recognition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[80]  Cocsku Oksuz,et al.  Ensemble-CVDNet: A Deep Learning based End-to-End Classification Framework for COVID-19 Detection using Ensembles of Networks , 2020, 2012.09132.

[81]  Lipo Wang,et al.  3D Deep Learning on Medical Images: A Review , 2020, Sensors.

[82]  Md. Kamrul Hasan,et al.  Brain Tissue Segmentation Using NeuroNet With Different Pre-processing Techniques , 2019, 2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR).

[83]  X. He,et al.  Benchmarking Deep Learning Models and Automated Model Design for COVID-19 Detection with Chest CT Scans , 2020, medRxiv.

[84]  Vivek Vaidya,et al.  Lung nodule detection in CT using 3D convolutional neural networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[85]  P. Xie,et al.  COVID-CT-Dataset: A CT Scan Dataset about COVID-19 , 2020, ArXiv.

[86]  Shaikh Anowarul Fattah,et al.  CovTANet: A Hybrid Tri-Level Attention-Based Network for Lesion Segmentation, Diagnosis, and Severity Prediction of COVID-19 Chest CT Scans , 2021, IEEE Transactions on Industrial Informatics.

[87]  Sharath Pankanti,et al.  Deep learning ensembles for melanoma recognition in dermoscopy images , 2016, IBM J. Res. Dev..

[88]  Md. Kamrul Hasan,et al.  DSNet: Automatic Dermoscopic Skin Lesion Segmentation , 2019, Comput. Biol. Medicine.

[89]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[90]  Y. Hu,et al.  Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China , 2020, The Lancet.

[91]  Zhao Zhang,et al.  CNN-based encoder-decoder networks for salient object detection: A comprehensive review and recent advances , 2021, Inf. Sci..

[92]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[93]  Deepak Gupta,et al.  CovidGAN: Data Augmentation Using Auxiliary Classifier GAN for Improved Covid-19 Detection , 2020, IEEE Access.

[94]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[95]  Md. Kamrul Hasan,et al.  DRNet: Segmentation and localization of optic disc and Fovea from diabetic retinopathy image , 2020, Artif. Intell. Medicine.

[96]  Kemal Polat,et al.  A Novel Medical Diagnosis model for COVID-19 infection detection based on Deep Features and Bayesian Optimization , 2020, Applied Soft Computing.

[97]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[98]  Ronald M. Summers,et al.  TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-Rays , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[99]  Sang Won Yoon,et al.  A 3D Convolutional Neural Network for Volumetric Image Semantic Segmentation , 2019, Procedia Manufacturing.

[100]  Qiang Li,et al.  Automated segmentation of lungs with severe interstitial lung disease in CT. , 2009, Medical physics.

[101]  Yabo Fu,et al.  Deep Learning in Medical Image Registration: A Review , 2020, Physics in medicine and biology.

[102]  Woo Kyung Moon,et al.  Combining support vector machine with genetic algorithm to classify ultrasound breast tumor images , 2012, Comput. Medical Imaging Graph..

[103]  Quoc V. Le,et al.  Efficient Neural Architecture Search via Parameter Sharing , 2018, ICML.

[104]  Ioannis D. Apostolopoulos,et al.  Covid-19: automatic detection from X-ray images utilizing transfer learning with convolutional neural networks , 2020, Physical and Engineering Sciences in Medicine.

[105]  Asif Iqbal Khan,et al.  CoroNet: A deep neural network for detection and diagnosis of COVID-19 from chest x-ray images , 2020, Computer Methods and Programs in Biomedicine.

[106]  Dilber Uzun Ozsahin,et al.  Review on Diagnosis of COVID-19 from Chest CT Images Using Artificial Intelligence , 2020, Comput. Math. Methods Medicine.

[107]  Daniel L. Rubin,et al.  Differential Data Augmentation Techniques for Medical Imaging Classification Tasks , 2017, AMIA.

[108]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[109]  Kaijin Xu,et al.  A Deep Learning System to Screen Novel Coronavirus Disease 2019 Pneumonia , 2020, Engineering.

[110]  Bal'azs Maga,et al.  Attention U-Net Based Adversarial Architectures for Chest X-ray Lung Segmentation information , 2020, ADGN@ECAI.

[111]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[112]  U. Rajendra Acharya,et al.  Convolutional neural networks for multi-class brain disease detection using MRI images , 2019, Comput. Medical Imaging Graph..

[113]  Ezz El-Din Hemdan,et al.  COVIDX-Net: A Framework of Deep Learning Classifiers to Diagnose COVID-19 in X-Ray Images , 2020, ArXiv.

[114]  W. Liang,et al.  Clinically Applicable AI System for Accurate Diagnosis, Quantitative Measurements, and Prognosis of COVID-19 Pneumonia Using Computed Tomography , 2020, Cell.

[115]  Margrit Betke,et al.  Chest CT: automated nodule detection and assessment of change over time--preliminary experience. , 2001, Radiology.

[116]  Qing Liu,et al.  Accurate Object Localization in Remote Sensing Images Based on Convolutional Neural Networks , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[117]  R. Kulkarni,et al.  Convolutional neural networks in medical image understanding: a survey , 2021, Evolutionary Intelligence.

[118]  Victor M Corman,et al.  Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR , 2020, Euro surveillance : bulletin Europeen sur les maladies transmissibles = European communicable disease bulletin.

[119]  Farayi Musharavati,et al.  Detection and Severity Classification of COVID-19 in CT Images Using Deep Learning , 2021, Diagnostics.

[120]  Josien P. W. Pluim,et al.  Not‐so‐supervised: A survey of semi‐supervised, multi‐instance, and transfer learning in medical image analysis , 2018, Medical Image Anal..

[121]  Andrew Y. Ng,et al.  CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning , 2017, ArXiv.

[122]  Umut Ozkaya,et al.  Coronavirus (COVID-19) Classification Using Deep Features Fusion and Ranking Technique , 2020, Big Data Analytics and Artificial Intelligence Against COVID-19: Innovation Vision and Approach.

[123]  Jing Zhao,et al.  Early Transmission Dynamics in Wuhan, China, of Novel Coronavirus–Infected Pneumonia , 2020, The New England journal of medicine.

[124]  Song Wang,et al.  Performance evaluation of 2D and 3D deep learning approaches for automatic segmentation of multiple organs on CT images , 2018, Medical Imaging.

[125]  C. L. Chowdhary,et al.  Deep learning and medical image processing for coronavirus (COVID-19) pandemic: A survey , 2020, Sustainable Cities and Society.

[126]  Sameer K. Antani,et al.  Iteratively Pruned Deep Learning Ensembles for COVID-19 Detection in Chest X-Rays , 2020, IEEE Access.

[127]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[128]  Elahe Arani,et al.  RGPNet: A Real-Time General Purpose Semantic Segmentation , 2019, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[129]  Mesut Toğaçar,et al.  COVID-19 detection using deep learning models to exploit Social Mimic Optimization and structured chest X-ray images using fuzzy color and stacking approaches , 2020, Computers in Biology and Medicine.

[130]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .