Performance analysis of No-Propagation and ELM algorithms in classification

The growing volume and complexity of data has led to the development of the so-called linear algorithms for neural networks like ELM, which maintain the precision of classic algorithms but with higher training speed. This speed increase is due to a simpler architecture, the random fixing of the input weights without being trained and the analytical calculation of the output weights instead of the slowly classical iterative gradient methods as Backpropagation. However, the random fixing of the input weights increases the sensibility to input perturbations like noise. Recently, No-Propagation (No-Prop) algorithm has been introduced as another linear algorithm, which shares with ELM the architecture and the random input weights (hidden weights) initialization. In this paper, an exhaustive comparison of both algorithms and its regularized versions are presented. The simulations results suggest that No-Prop is a competitive alternative to the ELM algorithm.

[1]  Andrés Bueno-Crespo,et al.  Neural architecture design based on extreme learning machine , 2013, Neural Networks.

[2]  Bernard Widrow.  Reply to the Comments on the "No-Prop" algorithm , 2013, Neural Networks.

[3]  L. P. Wang,et al.  Comments on "The Extreme Learning Machine" , 2008, IEEE Trans. Neural Networks.

[4]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[5]  Bernard Widrow,et al.  The No-Prop algorithm: A new learning algorithm for multilayer neural networks , 2013, Neural Networks.

[6]  Amaury Lendasse,et al.  Regularized extreme learning machine for regression with missing data , 2013, Neurocomputing.

[7]  Yong Yu,et al.  Sales forecasting using extreme learning machine with applications in fashion retailing , 2008, Decis. Support Syst..

[8]  Zhihong Man,et al.  Robust Single-Hidden Layer Feedforward Network-Based Pattern Classifier , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[9]  N. S. Marimuthu,et al.  Intelligent approaches using support vector machine and extreme learning machine for transmission line protection , 2010, Neurocomputing.

[10]  Amaury Lendasse,et al.  OP-ELM: Optimally Pruned Extreme Learning Machine , 2010, IEEE Transactions on Neural Networks.

[11]  José David Martín-Guerrero,et al.  Regularized extreme learning machine for regression problems , 2011, Neurocomputing.

[12]  Zhihong Man,et al.  A new robust training algorithm for a class of single-hidden layer feedforward neural networks , 2011, Neurocomputing.

[13]  Punyaphol Horata,et al.  Robust extreme learning machine , 2013, Neurocomputing.

[14]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[15]  Guang-Bin Huang,et al.  Trends in extreme learning machines: A review , 2015, Neural Networks.