Detecting brain tumors using deep learning convolutional neural network with transfer learning approach

Accurate classification of brain tumor subtypes is important for prognosis and treatment. In this study, we optimized and applied non‐deep learning methods based on hand‐crafted features and deep learning methods based on transfer learning using softmax as classification and KNN and SVM as classification for features extracted from deep features of ResNet101. For non‐deep learning techniques, we extracted multimodal features as input to machine learning classifiers. For convolutional neural networks, we optimized and applied GoogleNet and ResNet101with transfer learning approach. The performance was evaluated in terms of sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), false positive rate (FPR), total accuracy (TA), and area under the receiver operating curve (AUC) using Jack‐knife 10‐fold cross validation (CV) for the testing and validation of the dataset. For two‐class classification, entropy features using SVM Gaussian yielded the highest performance with 93.84% TA and 0.9874 AUC, and GoogleNet yielded 99.33% TA. For Multiclass classification, the highest performance to detect pituitary tumor yielded 95.65% accuracy and 0.95 AUC using ResNet101 with transfer learning. Deep features from ResNet101 using KNN improved detection of pituitary tumor (98.80% accuracy, 0.99 AUC), glioma (93.47% accuracy, 0.93 AUC), and meningioma (93.36% accuracy, 0.89 AUC). The deep features ResNet101‐SVM to detect pituitary tumor yielded performance (98.69% accuracy, 0.98 AUC). Deep learning methods with transfer learning along with softmax and KNN and SVM as classification outperformed traditional machine learning methods. This approach may prove useful for prognosis and treatment planning to achieve better clinical outcomes.

[1]  Ghazanfar Farooq Siddiqui,et al.  Prostate Cancer Detection Using Deep Learning and Traditional Techniques , 2021, IEEE Access.

[2]  Ronald M. Summers,et al.  Interleaved text/image Deep Mining on a large-scale radiology database , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  M Masseroli,et al.  Quantitative morphology and shape classification of neurons by computerized image analysis. , 1993, Computer methods and programs in biomedicine.

[4]  Konstantinos N. Plataniotis,et al.  Brain Tumor Type Classification via Capsule Networks , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[5]  E Le Rumeur,et al.  MRI texture analysis on texture test objects, normal brain and intracranial tumors. , 2003, Magnetic resonance imaging.

[6]  Mahmoud Khaled Abd-Ellah,et al.  Design and implementation of a computer-aided diagnosis system for brain tumor classification , 2016, 2016 28th International Conference on Microelectronics (ICM).

[7]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[8]  Charles R. Giardina,et al.  Elliptic Fourier features of a closed contour , 1982, Comput. Graph. Image Process..

[9]  Sebastian Thrun,et al.  Text Classification from Labeled and Unlabeled Documents using EM , 2000, Machine Learning.

[10]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  S. Fazli,et al.  Particle Filter Based Object Tracking with Sift and Color Feature , 2009, 2009 Second International Conference on Machine Vision.

[12]  Asmita Moghe,et al.  Brain tumor detection and classification in MRI images using image and data mining , 2017, 2017 International Conference on Recent Innovations in Signal processing and Embedded Systems (RISE).

[13]  Carles Arús,et al.  Automated classification of short echo time in in vivo 1H brain tumor spectra: A multicenter study , 2003, Magnetic resonance in medicine.

[14]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Ruwan D. Nawarathna,et al.  A sophisticated convolutional neural network model for brain tumor classification , 2017, 2017 IEEE International Conference on Industrial and Information Systems (ICIIS).

[16]  Anh-Cang Phan,et al.  Automatic Identification of Intracranial Hemorrhage on CT/MRI Image Using Meta-Architectures Improved from Region-Based CNN , 2019, WCGO.

[17]  Jitendra Malik,et al.  Learning Rich Features from RGB-D Images for Object Detection and Segmentation , 2014, ECCV.

[18]  Long Jin,et al.  Convolutional Neural Network Based on Complex Networks for Brain Tumor Image Classification With a Modified Activation Function , 2020, IEEE Access.

[19]  Tati L. R. Mengko,et al.  Brain Tumor Classification Using Convolutional Neural Network , 2018, IFMBE Proceedings.

[20]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[21]  Fang Zhang,et al.  Deep convolutional activation features for large scale Brain Tumor histopathology image classification and segmentation , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[22]  Hao Chen,et al.  Standard Plane Localization in Fetal Ultrasound via Domain Transferred Deep Neural Networks , 2015, IEEE Journal of Biomedical and Health Informatics.

[23]  Sung Wook Baik,et al.  Multi-grade brain tumor classification using deep CNN with extensive data augmentation , 2019, J. Comput. Sci..

[24]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[25]  Antoanela Naaji,et al.  A Modified Deep Convolutional Neural Network for Abnormal Brain Image Classification , 2019, IEEE Access.

[26]  Sabine Van Huffel,et al.  Brain tumor classification based on long echo proton MRS signals , 2004, Artif. Intell. Medicine.

[27]  Asifullah Khan,et al.  Automated colon cancer detection using hybrid of novel geometric features and some traditional features , 2015, Comput. Biol. Medicine.

[28]  P Babu Anto,et al.  Tumor detection and classification of MRI brain image using wavelet transform and SVM , 2017, 2017 International Conference on Signal Processing and Communication (ICSPC).

[29]  Patrick M. Pilarski,et al.  First steps towards an intelligent laser welding architecture using deep neural networks and reinforcement learning , 2014 .

[30]  A. W. Simonetti,et al.  The use of multivariate MR imaging intensities versus metabolic data from MR spectroscopic imaging for brain tumour classification. , 2005, Journal of magnetic resonance.

[31]  Feng Liu,et al.  Deep Learning and Its Applications in Biomedicine , 2018, Genom. Proteom. Bioinform..

[32]  M. Sornam,et al.  Segmentation and classification of brain tumor using wavelet and Zernike based features on MRI , 2016, 2016 IEEE International Conference on Advances in Computer Applications (ICACA).

[33]  L. Schad,et al.  MR image texture analysis--an approach to tissue characterization. , 1993, Magnetic resonance imaging.

[34]  Chandan Saha,et al.  A new hybrid approach for brain tumor classification using BWT-KSVM , 2017, 2017 4th International Conference on Advances in Electrical Engineering (ICAEE).

[35]  Christoph Meinel,et al.  Deep Neural Network with l2-Norm Unit for Brain Lesions Detection , 2017, ICONIP.

[36]  Suresh Chandra Satapathy,et al.  Gastrointestinal diseases segmentation and classification based on duo-deep architectures , 2020, Pattern Recognit. Lett..

[37]  Bolei Zhou,et al.  Learning Deep Features for Scene Recognition using Places Database , 2014, NIPS.

[38]  A. Diacon,et al.  Xpert MTB/RIF Assay for Diagnosis of Pleural Tuberculosis , 2011, Journal of Clinical Microbiology.

[39]  Anthony Maida,et al.  Natural Image Bases to Represent Neuroimaging Data , 2013, ICML.

[40]  R. Hristovski,et al.  From microscopic to macroscopic sports injuries. Applying the complex dynamic systems approach to sports medicine: a narrative review , 2018, British Journal of Sports Medicine.

[41]  Konstantinos P. Ferentinos,et al.  Deep learning models for plant disease detection and diagnosis , 2018, Comput. Electron. Agric..

[42]  Abdul Jalil,et al.  Ensemble classification of colon biopsy images based on information rich hybrid features , 2014, Comput. Biol. Medicine.

[43]  Vasiliki Kosmidou,et al.  Using sample entropy for automated sign language recognition on sEMG and accelerometer data , 2010, Medical & Biological Engineering & Computing.

[44]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[46]  Ahmad Kamran Malik,et al.  A multi‐modal, multi‐atlas‐based approach for Alzheimer detection via machine learning , 2018, Int. J. Imaging Syst. Technol..

[47]  Yong-Ming Li,et al.  A new strategy for urinary sediment segmentation based on wavelet, morphology and combination method , 2006, Comput. Methods Programs Biomed..

[48]  Hesham A. Hefny,et al.  An enhanced deep learning approach for brain cancer MRI images classification using residual networks , 2020, Artif. Intell. Medicine.

[49]  Abdul Jalil,et al.  Capture Largest Included Circles: An Approach for Counting Red Blood Cells , 2012 .

[50]  Ghazanfar Latif,et al.  Multiclass brain Glioma tumor classification using block-based 3D Wavelet features of MR images , 2017, 2017 4th International Conference on Electrical and Electronic Engineering (ICEEE).

[51]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[52]  Namita Mittal,et al.  Performance analysis of Gabor-Wavelet based features in classification of high grade malignant brain tumors , 2015, 2015 39th National Systems Conference (NSC).

[53]  Lal Hussain,et al.  Detecting epileptic seizure with different feature extracting strategies using robust machine learning classification techniques by applying advance parameter optimization approach , 2018, Cognitive Neurodynamics.

[54]  W. R. Sam Emmanuel,et al.  MRI Brain Tumor Classification Using Cuckoo Search Support Vector Machines and Particle Swarm Optimization Based Feature Selection , 2018 .

[55]  G. Ramani,et al.  MR Brain Tumor Classification and Segmentation Via Wavelets , 2018, 2018 International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET).

[56]  Ahmet Demir,et al.  Early detection of skin cancer using deep learning architectures: resnet-101 and inception-v3 , 2019, 2019 Medical Technologies Congress (TIPTEKNO).

[57]  P. Baldi,et al.  Deep-Learning Convolutional Neural Networks Accurately Classify Genetic Mutations in Gliomas , 2018, American Journal of Neuroradiology.

[58]  Gi-Hwan Choi,et al.  (1)H-MRS metabolic patterns for distinguishing between meningiomas and other brain tumors. , 2003, Magnetic resonance imaging.

[59]  J. Suykens,et al.  Classification of brain tumours using short echo time 1H MR spectra. , 2004, Journal of magnetic resonance.

[60]  Amjad Ali,et al.  Detecting Congestive Heart Failure by Extracting Multimodal Features and Employing Machine Learning Techniques , 2020, BioMed research international.

[61]  Victor Alves,et al.  Understanding and Interpreting Machine Learning in Medical Image Computing Applications , 2018, Lecture Notes in Computer Science.

[62]  Chandrakant Mahobiya,et al.  MR Image Classification Using Adaboost for Brain Tumor Type , 2017, 2017 IEEE 7th International Advance Computing Conference (IACC).

[63]  V. Balamurugan,et al.  Robust classification of primary brain tumor in Computer Tomography images using K-NN and linear SVM , 2014, 2014 International Conference on Contemporary Computing and Informatics (IC3I).

[64]  S.N. Deepa,et al.  Neural networks and SMO based classification for brain tumor , 2011, 2011 World Congress on Information and Communication Technologies.

[65]  A. Prabin,et al.  Robust Classification of Primary Brain Tumor in MRI Images Based on Multi Model Textures Features and Kernel Based SVM , 2015 .

[66]  Daniel Fabbri,et al.  Deep learning for brain tumor classification , 2017, Medical Imaging.

[67]  Saeed Arif Shah,et al.  Prostate cancer detection using machine learning techniques by employing combination of features extracting strategies. , 2017, Cancer biomarkers : section A of Disease markers.

[68]  Qianjin Feng,et al.  Retrieval of Brain Tumors by Adaptive Spatial Pooling and Fisher Vector Representation , 2016, PloS one.

[69]  Abdul Majid,et al.  Detecting prostate cancer using deep learning convolution neural network with transfer learning approach , 2020, Cognitive Neurodynamics.

[70]  Ahmad Ali,et al.  A Recent Survey on Colon Cancer Detection Techniques , 2013, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[71]  Weimin Huang,et al.  Brain tumor grading based on Neural Networks and Convolutional Neural Networks , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[72]  Konstantinos N. Plataniotis,et al.  Capsule Networks for Brain Tumor Classification Based on MRI Images and Coarse Tumor Boundaries , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[73]  Vinod Kumar,et al.  Multiclass Brain Tumor Classification Using GA-SVM , 2011, 2011 Developments in E-systems Engineering.

[74]  Milica M. Badža,et al.  Classification of Brain Tumors from MRI Images Using a Convolutional Neural Network , 2020, Applied Sciences.

[75]  W El-Deredy,et al.  Tumour grading from magnetic resonance spectroscopy: a comparison of feature extraction with variable selection , 2003, Statistics in medicine.

[76]  Lal Hussain,et al.  Detecting Brain Tumor using Machines Learning Techniques Based on Different Features Extracting Strategies. , 2019, Current Medical Imaging Reviews.

[77]  P. M. Sanjeev Kumar,et al.  Computer aided diagnostic for cancer detection using MRI images of brain (Brain tumor detection and classification system) , 2016, 2016 IEEE Annual India Conference (INDICON).

[78]  I. Bialynicki-Birula,et al.  Uncertainty relations for information entropy in wave mechanics , 1975 .

[79]  Arshad Aziz,et al.  A new rectangular window based image cropping method for generalization of brain neoplasm classification systems , 2018, Int. J. Imaging Syst. Technol..

[80]  Samuel H. Hawkins,et al.  Deep Feature Transfer Learning in Combination with Traditional Features Predicts Survival Among Patients with Lung Adenocarcinoma , 2016, Tomography.

[81]  Jitendra Malik,et al.  Indoor Scene Understanding with RGB-D Images: Bottom-up Segmentation, Object Detection and Semantic Segmentation , 2015, International Journal of Computer Vision.

[82]  Christos Davatzikos,et al.  Classification of brain tumor type and grade using MRI texture and shape in a machine learning scheme , 2009, Magnetic resonance in medicine.

[83]  Qianjin Feng,et al.  Enhanced Performance of Brain Tumor Classification via Tumor Region Augmentation and Partition , 2015, PloS one.

[84]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[85]  Muhammad Rafique,et al.  Automated Breast Cancer Detection Using Machine Learning Techniques by Extracting Different Feature Extracting Strategies , 2018, 2018 17th IEEE International Conference On Trust, Security And Privacy In Computing And Communications/ 12th IEEE International Conference On Big Data Science And Engineering (TrustCom/BigDataSE).

[86]  P. M. Ameer,et al.  Brain tumor classification using deep CNN features via transfer learning , 2019, Comput. Biol. Medicine.

[87]  Tao Chen,et al.  Expert Systems With Applications , 2022 .

[88]  Madhuri S. Joshi,et al.  Brain Tumor Classification using Principal Component Analysis and Probabilistic Neural Network , 2015 .

[89]  L. Schad,et al.  MR tissue characterization of intracranial tumors by means of texture analysis. , 1993, Magnetic resonance imaging.

[90]  Ketan Machhale,et al.  MRI brain cancer classification using hybrid classifier (SVM-KNN) , 2015, 2015 International Conference on Industrial Instrumentation and Control (ICIC).

[91]  Qinghua Zhao,et al.  Brain tumor classification for MR images using transfer learning and fine-tuning , 2019, Comput. Medical Imaging Graph..

[92]  G. Pradeepini,et al.  Brain tumor classification using mixed method approach , 2017, 2017 International Conference on Information Communication and Embedded Systems (ICICES).

[93]  Hedieh Sajedi,et al.  Brain Tumor Classification via Convolutional Neural Network and Extreme Learning Machines , 2018, 2018 8th International Conference on Computer and Knowledge Engineering (ICCKE).