A Deep Feature Learning Model for Pneumonia Detection Applying a Combination of mRMR Feature Selection and Machine Learning Models

Abstract Pneumonia is one of the diseases that people may encounter in any period of their lives. Approximately 18% of infectious diseases are caused by pneumonia. This disease may result in death in the following stages. In order to diagnose pneumonia as a medical condition, lung X-ray images are routinely examined by the field experts in the clinical practice. In this study, lung X-ray images that are available for the diagnosis of pneumonia were used. The convolutional neural network was employed as feature extractor, and some of existing convolutional neural network models that are AlexNet, VGG-16 and VGG-19 were utilized so as to realize this specific task. Then, the number of deep features was reduced from 1000 to 100 by using the minimum redundancy maximum relevance algorithm for each deep model. Accordingly, we achieved 100 deep features from each deep model, and we combined these features so as to provide an efficient feature set consisting of totally 300 deep features. In this step of the experiment, this feature set was given as an input to the decision tree, k-nearest neighbors, linear discriminant analysis, linear regression, and support vector machine learning models. Finally, all models ensured promising results, especially linear discriminant analysis yielded the most efficient results with an accuracy of 99.41%. Consequently, the results point out that the deep features provided robust and consistent features for pneumonia detection, and minimum redundancy maximum relevance method was found a beneficial tool to reduce the dimension of the feature set.

[1]  Stephen Marshall,et al.  Activation Functions: Comparison of trends in Practice and Research for Deep Learning , 2018, ArXiv.

[2]  Peng Gang,et al.  Dimensionality reduction in deep learning for chest X-ray analysis of lung cancer , 2018, 2018 Tenth International Conference on Advanced Computational Intelligence (ICACI).

[3]  A. Chang,et al.  Improving the Diagnosis, Management, and Outcomes of Children with Pneumonia: Where are the Gaps? , 2013, Front. Pediatr..

[4]  Er Bao Peng,et al.  Image Processing Technology Research of On-Line Thread Processing , 2014 .

[5]  Mathias W. Pletz,et al.  Advances in the prevention, management, and treatment of community-acquired pneumonia , 2010, F1000Research.

[6]  Alexander J. Smola,et al.  Efficient mini-batch training for stochastic optimization , 2014, KDD.

[7]  Aboul Ella Hassanien,et al.  Linear discriminant analysis: A detailed tutorial , 2017, AI Commun..

[8]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[9]  Rahib H Abiyev,et al.  Deep Convolutional Neural Networks for Chest Diseases Detection , 2018, Journal of healthcare engineering.

[10]  Frank Hutter,et al.  SGDR: Stochastic Gradient Descent with Warm Restarts , 2016, ICLR.

[11]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Hao Wang,et al.  Generalized linear discriminant analysis based on euclidean norm for gait recognition , 2018, Int. J. Mach. Learn. Cybern..

[13]  Nuno M. Fonseca Ferreira,et al.  Classification of Images of Childhood Pneumonia using Convolutional Neural Networks , 2019, BIOIMAGING.

[14]  Zafer Cömert,et al.  Comparison of Machine Learning Techniques for Fetal Heart Rate Classification , 2017 .

[15]  Krupal S. Parikh,et al.  Support Vector Machine – A Large Margin Classifier to Diagnose Skin Illnesses , 2016 .

[16]  George R. Thoma,et al.  Visualizing and explaining deep learning predictions for pneumonia detection in pediatric chest radiographs , 2019, Medical Imaging.

[17]  S. Ewig,et al.  Community-Acquired Pneumonia in Adults. , 2017, Deutsches Arzteblatt international.

[18]  Daniel S. Kermany,et al.  Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning , 2018, Cell.

[19]  C.-C. Jay Kuo.  Understanding convolutional neural networks with a mathematical model , 2016, J. Vis. Commun. Image Represent..

[20]  Yang Wang,et al.  Applications of Support Vector Machine (SVM) Learning in Cancer Genomics. , 2018, Cancer genomics & proteomics.

[21]  Chris H. Q. Ding,et al.  Minimum Redundancy Feature Selection from Microarray Gene Expression Data , 2005, J. Bioinform. Comput. Biol..

[22]  Sargun Shashi B. Rana.  A Review of Medical Image Enhancement Techniques for Image Processing , 2011 .

[23]  Zafer Cömert,et al.  DCCMED-Net: Densely connected and concatenated multi Encoder-Decoder CNNs for retinal vessel extraction from fundus images. , 2019, Medical hypotheses.

[24]  Zuopeng Zhang,et al.  Impact of pneumonia and lung cancer on mortality of women with hypertension , 2016, Scientific Reports.

[25]  Ahmad B A Hassanat,et al.  Two-point-based binary search trees for accelerating big data classification using KNN , 2018, PloS one.

[26]  Junhao Wen,et al.  Fundus Image Classification Using VGG-19 Architecture with PCA and SVD , 2018, Symmetry.

[27]  Hamid R. Tizhoosh,et al.  Projectron – A Shallow and Interpretable Network for Classifying Medical Images , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[28]  Jayanth Koushik.  Understanding Convolutional Neural Networks , 2016, ArXiv.

[29]  Richard K. G. Do,et al.  Convolutional neural networks: an overview and application in radiology , 2018, Insights into Imaging.

[30]  D. Mollura,et al.  Segmentation and Image Analysis of Abnormal Lungs at CT: Current Approaches, Challenges, and Future Trends. , 2015, Radiographics : a review publication of the Radiological Society of North America, Inc.

[31]  Xiaofeng Zhu,et al.  Efficient kNN Classification With Different Numbers of Nearest Neighbors , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Hyunsun Park,et al.  Training Deep Neural Network in Limited Precision , 2018, ArXiv.

[33]  Michael Grass,et al.  Comparison of Deep Learning Approaches for Multi-Label Chest X-Ray Classification , 2018, Scientific Reports.

[34]  Keiron O'Shea,et al.  An Introduction to Convolutional Neural Networks , 2015, ArXiv.

[35]  Mesut TOĞAÇAR,et al.  DEEP LEARNING APPROACH FOR CLASSIFICATION OF BREAST CANCER , 2018, 2018 International Conference on Artificial Intelligence and Data Processing (IDAP).

[36]  Matthew Lavine,et al.  The Early Clinical X-Ray in the United States: Patient Experiences and Public Perceptions , 2012, Journal of the history of medicine and allied sciences.

[37]  Wei Zeng,et al.  Chest X-Ray Analysis of Tuberculosis by Deep Learning with Segmentation and Augmentation , 2018, 2018 IEEE 38th International Conference on Electronics and Nanotechnology (ELNANO).

[38]  Anastasios Tefas,et al.  Learning Bag-of-Features Pooling for Deep Convolutional Neural Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[39]  Min Zhang,et al.  Optimized Compression for Implementing Convolutional Neural Networks on FPGA , 2019, Electronics.

[40]  Sven Behnke,et al.  Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition , 2010, ICANN.

[41]  Matthew Michelson,et al.  A Deep Learning Method to Automatically Identify Reports of Scientifically Rigorous Clinical Research from the Biomedical Literature: Comparative Analytic Study , 2018, Journal of medical Internet research.

[42]  Hari Om,et al.  MCRMR: Maximum coverage and relevancy with minimal redundancy based multi-document summarization , 2019, Expert Syst. Appl..

[43]  Anjali Kulkarni,et al.  Classification of lung cancer stages on CT scan images using image processing , 2014, 2014 IEEE International Conference on Advanced Communications, Control and Computing Technologies.

[44]  Kaizhu Huang,et al.  Reducing and Stretching Deep Convolutional Activation Features for Accurate Image Classification , 2018, Cognitive Computation.

[45]  Zhonghua Chen,et al.  Comparison of machine learning methods for classifying mediastinal lymph node metastasis of non-small cell lung cancer from 18F-FDG PET/CT images , 2017, EJNMMI Research.

[46]  Marius-Christian Frunza,et al.  Support Vector Machines , 2016 .

[47]  Nishtha Hooda,et al.  Big Data Deep Learning Framework using Keras: A Case Study of Pneumonia Prediction , 2018, 2018 4th International Conference on Computing Communication and Automation (ICCCA).

[48]  Abdulkadir Sengur,et al.  Efficient deep features selections and classification for flower species recognition , 2019, Measurement.

[49]  Jürgen Gross,et al.  Linear Regression , 2003 .

[50]  Lacra Pavel,et al.  On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning , 2017, ArXiv.

[51]  Md. Kamrul Hasan,et al.  Linear regression-based feature selection for microarray data classification , 2015, Int. J. Data Min. Bioinform..

[52]  Li Li,et al.  Maximum relevance minimum common redundancy feature selection for nonlinear data , 2017, Inf. Sci..

[53]  Andrew R. Webb,et al.  Statistical Pattern Recognition , 1999 .

[54]  Dragica Radosav,et al.  Deep Learning and Medical Diagnosis: A Review of Literature , 2018, Multimodal Technol. Interact..

[55]  Zafer Cömert,et al.  Computer-aided diagnosis system combining FCN and Bi-LSTM model for efficient breast cancer detection from histopathological images , 2019, Appl. Soft Comput..

[56]  Fang Zhang,et al.  Deep convolutional activation features for large scale Brain Tumor histopathology image classification and segmentation , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[57]  Zafer Cömert,et al.  Fetal Hypoxia Detection Based on Deep Convolutional Neural Network with Transfer Learning Approach , 2018, CSOS.

[58]  Panayiotis E. Pintelas,et al.  A Weighted Voting Ensemble Self-Labeled Algorithm for the Detection of Lung Abnormalities from X-Rays , 2019, Algorithms.

[59]  Burhan Ergen,et al.  Subclass Separation of White Blood Cell Images Using Convolutional Neural Network Models , 2019, Elektronika ir Elektrotechnika.

[60]  Jérémie Jakubowicz,et al.  Deep Learning versus Conventional Machine Learning for Detection of Healthcare-Associated Infections in French Clinical Narratives , 2019, Methods of Information in Medicine.

[61]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[62]  Sonia Akter,et al.  Community acquired bacterial pneumonia: aetiology, laboratory detection and antibiotic susceptibility pattern. , 2014, The Malaysian journal of pathology.

[63]  Zhijian Song,et al.  Computer-aided detection in chest radiography based on artificial intelligence: a survey , 2018, BioMedical Engineering OnLine.

[64]  Yanhui Guo,et al.  NS-k-NN: Neutrosophic Set-Based k-Nearest Neighbors Classifier , 2017, Symmetry.

[65]  Clarimar José Coelho,et al.  Computer-aided diagnosis in chest radiography for detection of childhood pneumonia , 2008, Int. J. Medical Informatics.

[66]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[67]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.