Denoising Laplacian multi-layer extreme learning machine

Most of semi-supervised learning algorithms based on manifold regularization framework are surface learning algorithms, such as semi-supervised ELM (SS-ELM) and Laplacian smooth twin support vector machine (Lap-STSVM). Multi-layer extreme learning machine (ML-ELM) stacks extreme learning machine based auto encoder (ELM-AE) to create a multi-layer neural network. ML-ELM not only approximates the complicated function but also achieves fast training time. The outputs of ELM-AE are the same as inputs, which cannot guarantee the effectiveness of the learning feature representations. We put forward extreme learning machine based denoising auto encoder (ELM-DAE) which introduces local denoising criterion into ELM-AE and is used as the basic component for Denoising ML-ELM. Resembling ML-ELM, Denoising ML-ELM stacks ELM-DAE to create a deep network. And then we introduce manifold regularization into the model of Denoising ML-ELM and propose denoising Laplacian ML-ELM (Denoising Lap-ML-ELM). Denoising Lap-ML-ELM is more efficient than SS-ELM in classification and does not need to spend too much time. Experimental results show that Denoising ML-ELM and Denoising Lap-ML-ELM are effective learning algorithms.

[1]  Cheng Wu,et al.  Semi-Supervised and Unsupervised Extreme Learning Machines , 2014, IEEE Transactions on Cybernetics.

[2]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[3]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[4]  Xia Liu,et al.  Is Extreme Learning Machine Feasible? A Theoretical Assessment (Part I) , 2015, IEEE Trans. Neural Networks Learn. Syst..

[5]  Yiqiang Chen,et al.  Weighted extreme learning machine for imbalance learning , 2013, Neurocomputing.

[6]  Amaury Lendasse,et al.  TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization , 2011, Neurocomputing.

[7]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[8]  Yoshua Bengio,et al.  Exploring Strategies for Training Deep Neural Networks , 2009, J. Mach. Learn. Res..

[9]  Jürgen Schmidhuber,et al.  Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction , 2011, ICANN.

[10]  Dipankar Das,et al.  Enhanced SenticNet with Affective Labels for Concept-Based Opinion Mining , 2013, IEEE Intelligent Systems.

[11]  Yuan-Hai Shao,et al.  Laplacian smooth twin support vector machine for semi-supervised classification , 2013, International Journal of Machine Learning and Cybernetics.

[12]  Danwei Wang,et al.  Sparse Extreme Learning Machine for Classification , 2014, IEEE Transactions on Cybernetics.

[13]  Guang-Bin Huang,et al.  An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels , 2014, Cognitive Computation.

[14]  Deng Wan Research on Extreme Learning of Neural Networks , 2010 .

[15]  Qinyu. Zhu Extreme Learning Machine , 2013 .

[16]  Victor C. M. Leung,et al.  Extreme Learning Machines [Trends & Controversies] , 2013, IEEE Intelligent Systems.

[17]  Yicong Zhou,et al.  Extreme learning machine for ranking: Generalization analysis and applications , 2014, Neural Networks.

[18]  Hongming Zhou,et al.  Extreme Learning Machines [Trends & Controversies] , 2013 .

[19]  Shifei Ding,et al.  Extreme learning machine and its applications , 2013, Neural Computing and Applications.

[20]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[21]  Yiqiang Chen,et al.  SELM: Semi-supervised ELM with application in sparse calibrated location estimation , 2011, Neurocomputing.

[22]  Lin Chen,et al.  Research on Extreme Learning of Neural Networks: Research on Extreme Learning of Neural Networks , 2010 .

[23]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[24]  Amaury Lendasse,et al.  OP-ELM: Optimally Pruned Extreme Learning Machine , 2010, IEEE Transactions on Neural Networks.

[25]  Yiqiang Chen,et al.  Semi-supervised deep extreme learning machine for Wi-Fi based localization , 2015, Neurocomputing.

[26]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[27]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[28]  Honglak Lee,et al.  Unsupervised learning of hierarchical representations with convolutional deep belief networks , 2011, Commun. ACM.