Unsupervised kernel least mean square algorithm for solving ordinary differential equations

In this paper a novel method is introduced based on the use of an unsupervised version of kernel least mean square (KLMS) algorithm for solving ordinary differential equations (ODEs). The algorithm is unsupervised because here no desired signal needs to be determined by user and the output of the model is generated by iterating the algorithm progressively. However, there are several new approaches in literature to solve ODEs but the new approach has more advantages such as simple implementation, fast convergence and also little error. Furthermore, it is also a KLMS with obvious characteristics. In this paper the ability of KLMS is used to estimate the answer of ODE. First a trial solution of ODE is written as a sum of two parts, the first part satisfies the initial condition and the second part is trained using the KLMS algorithm so as the trial solution solves the ODE. The accuracy of the method is illustrated by solving several problems. Also the sensitivity of the convergence is analyzed by changing the step size parameters and kernel functions. Finally, the proposed method is compared with neuro-fuzzy [21] approach.

[1]  Weifeng Liu,et al.  The Kernel Least-Mean-Square Algorithm , 2008, IEEE Transactions on Signal Processing.

[2]  Nam Mai-Duy,et al.  Numerical solution of Navier–Stokes equations using multiquadric radial basis function networks , 2001 .

[3]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[4]  Mohsen Hayati,et al.  Multilayer perceptron neural networks with novel unsupervised training method for numerical solution of the partial differential equations , 2009, Appl. Soft Comput..

[5]  Jose C. Principe,et al.  Decoding hand trajectories from ECoG recordings via kernel least-mean-square algorithm , 2009, 2009 4th International IEEE/EMBS Conference on Neural Engineering.

[6]  N. Smaoui,et al.  Modelling the dynamics of nonlinear partial differential equations using neural networks , 2004 .

[7]  Sohrab Effati,et al.  Artificial neural network approach for solving fuzzy differential equations , 2010, Inf. Sci..

[8]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[9]  Weifeng Liu,et al.  Kernel LMS , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[10]  Hua Bao,et al.  Active Noise Control based on Kernel Least-Mean-Square algorithm , 2009, 2009 Conference Record of the Forty-Third Asilomar Conference on Signals, Systems and Computers.

[11]  Hadi Sadoghi Yazdi,et al.  A new modeling algorithm - Normalized Kernel Least Mean Square , 2009, 2009 International Conference on Innovations in Information Technology (IIT).

[12]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[13]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[14]  Euripidis Glavas,et al.  Solving differential equations with constructed neural networks , 2009, Neurocomputing.

[15]  Konrad Reif,et al.  Multilayer neural networks for solving a class of partial differential equations , 2000, Neural Networks.

[16]  Hadi Sadoghi Yazdi,et al.  Kernel Least Mean Square Features for HMM-Based Signal Recognition , 2010 .

[17]  Siwei Luo,et al.  Numerical solution of elliptic partial differential equation by growing radial basis function neural networks , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[18]  T. Leephakpreeda.  Novel determination of differential-equation solutions: universal approximation method , 2002 .

[19]  Alexander J. Smola,et al.  Online learning with kernels , 2001, IEEE Transactions on Signal Processing.

[20]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[21]  Alaeddin Malek,et al.  Numerical solution for high order differential equations using a hybrid neural network - Optimization method , 2006, Appl. Math. Comput..

[22]  B. Widrow,et al.  Adaptive noise cancelling: Principles and applications , 1975 .

[23]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[24]  Ming Gen Cui,et al.  New algorithm for a class of nonlinear integro-differential equations in the reproducing kernel space , 2006, Appl. Math. Comput..

[25]  D. Parisi,et al.  Solving differential equations with unsupervised neural networks , 2003 .

[26]  S. Haykin.  Adaptive Filters , 2007 .

[27]  F. Diebold,et al.  Comparing Predictive Accuracy , 1994, Business Cycles.

[28]  Nam Mai-Duy,et al.  Numerical solution of differential equations using multiquadric radial basis function networks , 2001, Neural Networks.

[29]  Hadi Sadoghi Yazdi,et al.  Unsupervised adaptive neural-fuzzy inference system for solving differential equations , 2010, Appl. Soft Comput..

[30]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003 .

[31]  Rüdiger W. Brause.  Adaptive Modeling of Biochemical Pathways , 2004, Int. J. Artif. Intell. Tools.

[32]  Paul Newbold,et al.  Testing the equality of prediction mean squared errors , 1997 .

[33]  Ali H. Sayed,et al.  Adaptive Filters , 2008 .

[34]  Dan Givoli,et al.  Neural network time series forecasting of finite-element mesh adaptation , 2005, Neurocomputing.

[35]  Luo Siwei,et al.  Numerical solution of elliptic partial differential equation using radial basis function neural networks. , 2003, Neural networks : the official journal of the International Neural Network Society.

[36]  Alaeddin Malek,et al.  Solving initial-boundary value problems for systems of partial differential equations using neural networks and optimization techniques , 2009, J. Frankl. Inst..