Application of error minimized extreme learning machine for simultaneous learning of a function and its derivatives

Abstract In this paper a new learning algorithm is proposed for the problem of simultaneous learning of a function and its derivatives as an extension of the study of error minimized extreme learning machine for single hidden layer feedforward neural networks. Our formulation leads to solving a system of linear equations and its solution is obtained by Moore–Penrose generalized pseudo-inverse. In this approach the number of hidden nodes is automatically determined by repeatedly adding new hidden nodes to the network either one by one or group by group and updating the output weights incrementally in an efficient manner until the network output error is less than the given expected learning accuracy. For the verification of the efficiency of the proposed method a number of interesting examples are considered and the results obtained with the proposed method are compared with that of other two popular methods. It is observed that the proposed method is fast and produces similar or better generalization performance on the test data.

[1]  Robert K. L. Gay,et al.  Error Minimized Extreme Learning Machine With Growth of Hidden Nodes and Incremental Learning , 2009, IEEE Transactions on Neural Networks.

[2]  Lucie P. Aarts,et al.  Neural Network Method for Solving Partial Differential Equations , 2001, Neural Processing Letters.

[3]  César Caballero-Gaudes,et al.  Robust blind identification of SIMO channels: a support vector regression approach , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[4]  Leszek Rutkowski,et al.  A general approach for nonparametric fitting of functions and their derivatives with applications to linear circuits identification , 1986 .

[5]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[6]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[7]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.

[8]  T. Nguyen-Thien,et al.  Approximation of functions and their derivatives: A neural network implementation with applications , 1999 .

[9]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[10]  F. Tay,et al.  Application of support vector machines in financial time series forecasting , 2001 .

[11]  Xin Li,et al.  Simultaneous approximations of multivariate functions and their derivatives by neural networks with one hidden layer , 1996, Neurocomputing.

[12]  Lei Chen,et al.  Enhanced random search based incremental extreme learning machine , 2008, Neurocomputing.

[13]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[14]  Hongming Zhou,et al.  Optimization method based extreme learning machine for classification , 2010, Neurocomputing.

[15]  Piet M. T. Broersen.  How to select polynomial models with an accurate derivative , 2000, IEEE Trans. Instrum. Meas..

[16]  Reshma Khemchandani,et al.  Regularized least squares support vector regression for the simultaneous learning of a function and its derivatives , 2008, Inf. Sci..

[17]  Fernando Pérez-Cruz,et al.  Support Vector Regression for the simultaneous learning of a multivariate function and its derivatives , 2005, Neurocomputing.

[18]  K. S. Banerjee.  Generalized Inverse of Matrices and Its Applications , 1973 .