Ensemble RBM-based classifier using fuzzy integral for big data classification

The restricted Boltzmann machine (RBM) is a primary building block of deep learning models. As an efficient representation learning approach, deep RBM can effectively extract sophisticated and informative features from raw data. Little research has been undertaken on using deep RBM to extract features from big data however. In this paper, we investigate this problem, and an ensemble approach for big data classification based on Hadoop MapReduce and fuzzy integral is proposed. The proposed method consists of two stages, map and reduce. In the map stage, multiple RBM-based classifiers used for ensemble are trained in parallel. In the reduce stage, the trained multiple RBM-based classifiers are integrated by fuzzy integral. Experiments on five big data sets show that the proposed approach can outperform other baseline methods to achieve state-of-the-art performance.

[1]  Razvan Pascanu,et al.  Learning Algorithms for the Classification Restricted Boltzmann Machine , 2012, J. Mach. Learn. Res..

[2]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[3]  Christian Igel,et al.  Training restricted Boltzmann machines: An introduction , 2014, Pattern Recognit..

[4]  Chun-Xia Zhang,et al.  Learning ensemble classifiers via restricted Boltzmann machines , 2014, Pattern Recognit. Lett..

[5]  Jiwen Lu,et al.  Learning Cascaded Deep Auto-Encoder Networks for Face Alignment , 2016, IEEE Transactions on Multimedia.

[6]  Geoffrey E. Hinton,et al.  Factored 3-Way Restricted Boltzmann Machines For Modeling Natural Images , 2010, AISTATS.

[7]  Xi-Zhao Wang,et al.  Improving Generalization of Fuzzy IF--THEN Rules by Maximizing Fuzzy Entropy , 2009, IEEE Transactions on Fuzzy Systems.

[8]  Wen Yu,et al.  Deep Boltzmann machine for nonlinear system modelling , 2019, Int. J. Mach. Learn. Cybern..

[9]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[10]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[11]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[12]  Enrique Romero Merino,et al.  Neighborhood-Based Stopping Criterion for Contrastive Divergence , 2018, IEEE Trans. Neural Networks Learn. Syst..

[13]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Xiao-Jun Wu,et al.  Multiple birth least squares support vector machine for multi-class classification , 2017, Int. J. Mach. Learn. Cybern..

[15]  Zhongzhi Shi,et al.  Unsupervised extreme learning machine with representational features , 2015, International Journal of Machine Learning and Cybernetics.

[16]  Kenji Doya,et al.  Expected energy-based restricted Boltzmann machine for classification , 2015, Neural Networks.

[17]  Geoffrey E. Hinton,et al.  Using fast weights to improve persistent contrastive divergence , 2009, ICML '09.

[18]  Chokri Ben Amar,et al.  Statistical binary patterns and post-competitive representation for pattern recognition , 2018, Int. J. Mach. Learn. Cybern..

[19]  Xue-wen Chen,et al.  Big Data Deep Learning: Challenges and Perspectives , 2014, IEEE Access.

[20]  Witold Pedrycz,et al.  A Study on Relationship Between Generalization Abilities and Fuzziness of Base Classifiers in Ensemble Learning , 2015, IEEE Transactions on Fuzzy Systems.

[21]  Yoshua Bengio,et al.  Classification using discriminative restricted Boltzmann machines , 2008, ICML '08.

[22]  Tijmen Tieleman,et al.  Training restricted Boltzmann machines using approximations to the likelihood gradient , 2008, ICML '08.

[23]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[24]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[25]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[26]  Hongming Zhou,et al.  Representational Learning with ELMs for Big Data , 2013 .

[27]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[28]  Ran Wang,et al.  Noniterative Deep Learning: Incorporating Restricted Boltzmann Machine Into Multilayer Random Weight Neural Networks , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[29]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[30]  Xizhao Wang,et al.  Nested structure in parameterized rough reduction , 2013, Inf. Sci..

[31]  Junyu Dong,et al.  An Overview on Data Representation Learning: From Traditional Feature Learning to Recent Deep Learning , 2016, ArXiv.

[32]  Geng Yang,et al.  Local mean representation based classifier and its applications for data classification , 2018, Int. J. Mach. Learn. Cybern..

[33]  Laurence T. Yang,et al.  A survey on deep learning for big data , 2018, Inf. Fusion.

[34]  C. L. Philip Chen,et al.  A Fuzzy Restricted Boltzmann Machine: Novel Learning Algorithms Based on the Crisp Possibilistic Mean Value of Fuzzy Numbers , 2018, IEEE Transactions on Fuzzy Systems.

[35]  Shifei Ding,et al.  An overview on Restricted Boltzmann Machines , 2018, Neurocomputing.

[36]  Geoffrey E. Hinton.  Products of experts , 1999 .

[37]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[38]  Jian Xu,et al.  Symmetrical singular value decomposition representation for pattern recognition , 2016, Neurocomputing.

[39]  Xue-wen Chen,et al.  Large-Scale Deep Belief Nets With MapReduce , 2014, IEEE Access.

[40]  Guowei Yang,et al.  Local descriptor margin projections (LDMP) for face recognition , 2018, Int. J. Mach. Learn. Cybern..

[41]  Ping Wang,et al.  A fast and efficient conformal regressor with regularized extreme learning machine , 2018, Neurocomputing.

[42]  Zhang Yi,et al.  Graph Regularized Restricted Boltzmann Machine , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[43]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[44]  Dewang Chen,et al.  MapReduce based distributed learning algorithm for Restricted Boltzmann Machine , 2016, Neurocomputing.

[45]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[46]  Lili Gan,et al.  Experimental study on generalization capability of extended naive Bayesian classifier , 2014, International Journal of Machine Learning and Cybernetics.

[47]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[48]  Jianping Fan,et al.  Least squares kernel ensemble regression in Reproducing Kernel Hilbert Space , 2018, Neurocomputing.