论文信息 - A Vest of the Pseudoinverse Learning Algorithm

A Vest of the Pseudoinverse Learning Algorithm

In this paper, we briefly review the basic scheme of the pseudoinverse learning (PIL) algorithm and present some discussions on the PIL, as well as its variants. The PIL algorithm, first presented in 1995, is a non-gradient descent and non-iterative learning algorithm for multi-layer neural networks and has several advantages compared with gradient descent based algorithms. Some new viewpoints to PIL algorithm are presented, and several common pitfalls in practical implementation of the neural network learning task are also addressed. In addition, we show that so called extreme learning machine is a Variant crEated by Simple name alTernation (VEST) of the PIL algorithm for single hidden layer feedforward neural networks.

Ping Guo | Ping Guo

[1] Qinyu. Zhu. Extreme Learning Machine , 2013 .

[2] Geoffrey E. Hinton,et al. Learning internal representations by error propagation , 1986 .

[3] Michael R. Lyu,et al. A pseudoinverse learning algorithm for feedforward neural networks with stacked generalization applications to software reliability growth data , 2004, Neurocomputing.

[4] Heekuck Oh,et al. Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[5] Christopher M. Bishop,et al. Current address: Microsoft Research, , 2022 .

[6] Yoshua Bengio,et al. Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[7] Chee Kheong Siew,et al. Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[8] Ming Li,et al. Insights into randomized algorithms for neural networks: Practical issues and common pitfalls , 2017, Inf. Sci..

[9] Guang-Bin Huang,et al. Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[10] C. L. Philip Chen,et al. Regularization parameter estimation for feedforward neural networks , 2003 .

[11] L. P. Wang,et al. Comments on "The Extreme Learning Machine" , 2008, IEEE Trans. Neural Networks.

[12] Etienne Barnard,et al. Avoiding false local minima by proper initialization of connections , 1992, IEEE Trans. Neural Networks.