An analysis of ELM approximate error based on random weight matrix

It is experimentally observed that the approximate errors of extreme learning machine (ELM) are dependent on the uniformity of training samples after the network architecture is fixed, and the uniformity, which is usually measured by the variance of distances among samples, varies with the linear transformation induced by the random weight matrix. By analyzing the dimension increase process in ELM, this paper gives an approximate relation between the uniformities before and after the linear transformation. Furthermore, by restricting ELM with a two-dimensional space, it gives an upper bound of ELM approximate error which is dependent on the distributive uniformity of training samples. The analytic results provide some useful guidelines to make clear the impact of random weights on ELM approximate ability and improve ELM prediction accuracy.

[1]  Korris Fu-Lai Chung,et al.  Positive and negative fuzzy rule system, extreme learning machine and image classification , 2011, Int. J. Mach. Learn. Cybern..

[2]  Weiming Cao,et al.  On the Error of Linear Interpolation and the Orientation, Aspect Ratio, and Internal Angles of a Triangle , 2005, SIAM J. Numer. Anal..

[3]  Daniel S. Yeung,et al.  Localized Generalization Error Model and Its Application to Architecture Selection for Radial Basis Function Neural Network , 2007, IEEE Transactions on Neural Networks.

[4]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[5]  Xizhao Wang,et al.  A comparative study on heuristic algorithms for generating fuzzy decision trees , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[6]  I. Babuska,et al.  ON THE ANGLE CONDITION IN THE FINITE ELEMENT METHOD , 1976 .

[7]  Albert Nigrin,et al.  Neural networks for pattern recognition , 1993 .

[8]  S. Rippa Long and thin triangles can be good for linear interpolation , 1992 .

[9]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[10]  Yadong Wang,et al.  A new approach to fuzzy rule generation: fuzzy extension matrix , 2001, Fuzzy Sets Syst..

[11]  J. Bramble,et al.  Triangular elements in the finite element method , 1970 .

[12]  Xizhao Wang,et al.  Improving learning accuracy of fuzzy decision trees by hybrid neural networks , 2000, IEEE Trans. Fuzzy Syst..

[13]  Xizhao Wang,et al.  A study on random weights between input and hidden layers in extreme learning machine , 2012, Soft Comput..

[14]  Xi-Zhao Wang,et al.  Learning optimization in simplifying fuzzy rules , 1999, Fuzzy Sets Syst..