Fuzzy Removing Redundancy Restricted Boltzmann Machine: Improving Learning Speed and Classification Accuracy

To improve the feature extraction ability and shorten the learning time, fuzzy removing redundancy restricted Boltzmann machine (F3RBM) is developed. The features extracted by F3RBM with unsupervised learning are imported into support vector machine (SVM) to establish F3RBM-SVM model, which achieves fast and high-precision automatic classification of different samples. To expand the feature extraction capability of restricted Boltzmann machine (RBM), the deterministic parameters of control model are replaced by fuzzy numbers in view of the superiority of fuzzy idea and the redundancy removal mechanism is introduced. Comparing the feature similarity of hidden units with the threshold value, if the similarity is greater than the threshold value, they are considered to be redundant units with the same features. The redundant units are removed to achieve further dimension reduction. Finally, the learning speed, feature extraction ability, and classification accuracy of different models are compared in MINIST handwritten dataset, Fashion MNIST dataset, and Olivetti Face dataset. The experimental results show that the feature extraction capability of FRBM and F3RBM is better than that of RBM. When there are a large number of hidden units, the learning speed of F3RBM is obviously faster than that of FRBM. The features extracted from F3RBM are imported into the SVM to build F3RBM-SVM model, which improves the classification accuracy and learning speed than general classifier. When adding other noises, F3RBM-SVM has better robustness than other models.

[1]  Jian Guo,et al.  Improved side information generation algorithm based on naive Bayesian theory for distributed video coding , 2018, IET Image Process..

[2]  Wei Zhou,et al.  Probability Calculation and Element Optimization of Probabilistic Hesitant Fuzzy Preference Relations Based on Expected Consistency , 2018, IEEE Transactions on Fuzzy Systems.

[3]  Petros Boufounos,et al.  A Deep Neural Network Architecture Using Dimensionality Reduction with Sparse Matrices , 2016, ICONIP.

[4]  Marc Winter,et al.  Objective microstructure classification by support vector machine (SVM) using a combination of morphological parameters and textural features for low carbon steels , 2019, Computational Materials Science.

[5]  Payman Moallem,et al.  Fuzzy Inference System Optimized by Genetic Algorithm for Robust Face and Pose Detection , 2015 .

[6]  Fernando Jiménez,et al.  Multiobjective Evolutionary Feature Selection for Fuzzy Classification , 2019, IEEE Transactions on Fuzzy Systems.

[7]  Hani Hagras,et al.  Multiobjective Evolutionary Optimization of Type-2 Fuzzy Rule-Based Systems for Financial Data Classification , 2017, IEEE Transactions on Fuzzy Systems.

[8]  Nasrollah Moghaddam Charkari,et al.  Survey on deep learning methods in human action recognition , 2017, IET Comput. Vis..

[9]  Dejan Dovzan,et al.  Implementation of an Evolving Fuzzy Model (eFuMo) in a Monitoring System for a Waste-Water Treatment Process , 2015, IEEE Transactions on Fuzzy Systems.

[10]  Hugo Larochelle,et al.  An Infinite Restricted Boltzmann Machine , 2015, Neural Computation.

[11]  C. L. Philip Chen,et al.  A Fuzzy Restricted Boltzmann Machine: Novel Learning Algorithms Based on the Crisp Possibilistic Mean Value of Fuzzy Numbers , 2018, IEEE Transactions on Fuzzy Systems.

[12]  F. Z. Laallam,et al.  Deep learning feature extraction for multispectral palmprint identification , 2018, J. Electronic Imaging.

[13]  Chunxiao Jiang,et al.  Microblog Dimensionality Reduction—A Deep Learning Approach , 2016, IEEE Transactions on Knowledge and Data Engineering.

[14]  Geoffrey E. Hinton.  Learning multiple layers of representation , 2007, Trends in Cognitive Sciences.

[15]  Manuel Berenguel,et al.  New approach for solar tracking systems based on computer vision, low cost hardware and deep learning , 2018, Renewable Energy.

[16]  Wei Jia,et al.  Parallel K Nearest Neighbor Matching for 3D Reconstruction , 2019, IEEE Access.

[17]  Xi Liu,et al.  A removing redundancy Restricted Boltzmann Machine , 2018, 2018 Tenth International Conference on Advanced Computational Intelligence (ICACI).

[18]  Ke Zhang,et al.  An optimized dimensionality reduction model for high-dimensional data based on Restricted Boltzmann Machines , 2015, The 27th Chinese Control and Decision Conference (2015 CCDC).

[19]  Hong Li,et al.  Quaternion-Based Multiscale Analysis for Feature Extraction of Hyperspectral Images , 2019, IEEE Transactions on Signal Processing.

[20]  Chin-Hui Lee,et al.  An End-to-End Deep Learning Approach to Simultaneous Speech Dereverberation and Acoustic Modeling for Robust Speech Recognition , 2017, IEEE Journal of Selected Topics in Signal Processing.

[21]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[22]  Feiping Nie,et al.  Rank-$k$ 2-D Multinomial Logistic Regression for Matrix Data Classification , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Tao Liu,et al.  Supervised Sparse Coding With Decision Forest , 2019, IEEE Signal Processing Letters.

[24]  Mohamed Farah,et al.  Hyperspectral imagery classification based on semi-supervised 3-D deep neural network and adaptive band selection , 2019, Expert Syst. Appl..

[25]  Patricia Melin,et al.  A hybrid modular neural network architecture with fuzzy Sugeno integration for time series forecasting , 2007, Appl. Soft Comput..

[26]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[27]  Radu-Emil Precup,et al.  Lorenz System Stabilization Using Fuzzy Controllers , 2007, Int. J. Comput. Commun. Control.

[28]  Honghai Liu,et al.  Jointly network: a network based on CNN and RBM for gesture recognition , 2018, Neural Computing and Applications.

[29]  Zheru Chi,et al.  Facial Expression Recognition in Video with Multiple Feature Fusion , 2018, IEEE Transactions on Affective Computing.

[30]  Geoffrey E. Hinton,et al.  Acoustic Modeling Using Deep Belief Networks , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[31]  Shuang Feng,et al.  Generative and Discriminative Fuzzy Restricted Boltzmann Machine Learning for Text and Image Classification , 2020, IEEE Transactions on Cybernetics.

[32]  Ming-Ai Li,et al.  A novel feature extraction method for scene recognition based on Centered Convolutional Restricted Boltzmann Machines , 2015, Neurocomputing.

[33]  C. L. Philip Chen,et al.  Fuzzy Restricted Boltzmann Machine for the Enhancement of Deep Learning , 2015, IEEE Transactions on Fuzzy Systems.

[34]  Francisco Herrera,et al.  A Metahierarchical Rule Decision System to Design Robust Fuzzy Classifiers Based on Data Complexity , 2019, IEEE Transactions on Fuzzy Systems.

[35]  Hirokazu Kameoka,et al.  Maximum reconstruction probability training of Restricted Boltzmann machines with auxiliary function approach , 2014, 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP).

[36]  Victor Hugo C. de Albuquerque,et al.  A novel electrocardiogram feature extraction approach for cardiac arrhythmia classification , 2019, Future Gener. Comput. Syst..

[37]  Timothy C. Havens,et al.  Efficient Multiple Kernel Classification Using Feature and Decision Level Fusion , 2017, IEEE Transactions on Fuzzy Systems.

[38]  Mostafa El Habib Daho,et al.  Combining Bootstrapping Samples, Random Subspaces and Random Forests to Build Classifiers , 2015 .

[39]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[40]  Eman M. G. Younis,et al.  Deep learning analysis of mobile physiological, environmental and location sensor data for emotion detection , 2019, Inf. Fusion.

[41]  S. Saravanan,et al.  An Enhancement of Deep Learning Algorithm for Brain Tumor Segmentation Using Kernel Based CNN with M-SVM , 2019, Journal of Medical Systems.

[42]  Chunlei Yang,et al.  From Class-Specific to Class-Mixture: Cascaded Feature Representations via Restricted Boltzmann Machine Learning , 2018, IEEE Access.

[43]  A. Meimandi.  Histopathological and biomechanical evaluation of bone healing properties of DBM and DBM-G90 in a rabbit model , 2015 .

[44]  Haizhou Li,et al.  Real-Time Keypoint Recognition Using Restricted Boltzmann Machine , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[45]  Jian Lü,et al.  Extension control strategy of a single converter for hybrid PEMFC/battery power source , 2018 .

[46]  Chen Chen,et al.  Deep Learning and Superpixel Feature Extraction Based on Contractive Autoencoder for Change Detection in SAR Images , 2018, IEEE Transactions on Industrial Informatics.

[47]  Hongxun Yao,et al.  Auto-encoder based dimensionality reduction , 2016, Neurocomputing.

[48]  Mohammed Bennamoun,et al.  Learning Non-linear Reconstruction Models for Image Set Classification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Chao Qin,et al.  A comprehensive review on hybrid power system for PEMFC-HEV: Issues and strategies , 2018, Energy Conversion and Management.