Ordinary differential equations solution in kernel space

This paper presents a new method based on the use of an optimization approach along with kernel least mean square (KLMS) algorithm for solving ordinary differential equations (ODEs). The new approach in comparison with the other existing methods (such as numerical methods and the methods that are based on neural networks) has more advantages such as simple implementation, fast convergence, and also little error. In this paper, we use the ability of KLMS in prediction by applying an optimization method to predict the solution of ODE. The basic idea is that first a trial solution of the ODE is written by using the KLMS structure, and then by defining an error function and minimizing it via an optimization algorithm (in this paper, we used the quasi-Newton BFGS method), the parameters of KLMS are adjusted such that the trial solution satisfies the DE. After the optimization step, the achieved optimal parameters of the KLMS model are replaced in the trial solution. The accuracy of the method is illustrated by solving several problems.

[1]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[2]  Rüdiger W. Brause.  Adaptive modeling of biochemical pathways , 2003, Proceedings. 15th IEEE International Conference on Tools with Artificial Intelligence.

[3]  Weifeng Liu,et al.  Kernel LMS , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[4]  N. Smaoui,et al.  Modelling the dynamics of nonlinear partial differential equations using neural networks , 2004 .

[5]  D. Parisi,et al.  Solving differential equations with unsupervised neural networks , 2003 .

[6]  T. Leephakpreeda.  Novel determination of differential-equation solutions: universal approximation method , 2002 .

[7]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[8]  Nam Mai-Duy,et al.  Numerical solution of differential equations using multiquadric radial basis function networks , 2001, Neural Networks.

[9]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[10]  Sohrab Effati,et al.  Artificial neural network approach for solving fuzzy differential equations , 2010, Inf. Sci..

[11]  Konrad Reif,et al.  Multilayer neural networks for solving a class of partial differential equations , 2000, Neural Networks.

[12]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[13]  Dan Givoli,et al.  Neural network time series forecasting of finite-element mesh adaptation , 2005, Neurocomputing.

[14]  Siwei Luo,et al.  Numerical solution of elliptic partial differential equation using radial basis function neural networks , 2003, Neural Networks.

[15]  Jose C. Principe,et al.  Decoding hand trajectories from ECoG recordings via kernel least-mean-square algorithm , 2009, 2009 4th International IEEE/EMBS Conference on Neural Engineering.

[16]  S. Haykin,et al.  Kernel Least‐Mean‐Square Algorithm , 2010 .

[17]  Alaeddin Malek,et al.  Numerical solution for high order differential equations using a hybrid neural network - Optimization method , 2006, Appl. Math. Comput..

[18]  Luo Siwei,et al.  Numerical solution of elliptic partial differential equation using radial basis function neural networks. , 2003, Neural networks : the official journal of the International Neural Network Society.

[19]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[20]  Hadi Sadoghi Yazdi,et al.  Unsupervised kernel least mean square algorithm for solving ordinary differential equations , 2011, Neurocomputing.

[21]  Hadi Sadoghi Yazdi,et al.  Unsupervised adaptive neural-fuzzy inference system for solving differential equations , 2010, Appl. Soft Comput..

[22]  Rüdiger W. Brause.  Adaptive Modeling of Biochemical Pathways , 2004, Int. J. Artif. Intell. Tools.

[23]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.