Multilayer perceptrons and radial basis function neural network methods for the solution of differential equations: A survey

Since neural networks have universal approximation capabilities, therefore it is possible to postulate them as solutions for given differential equations that define unsupervised errors. In this paper, we present a wide survey and classification of different Multilayer Perceptron (MLP) and Radial Basis Function (RBF) neural network techniques, which are used for solving differential equations of various kinds. Our main purpose is to provide a synthesis of the published research works in this area and stimulate further research interest and effort in the identified topics. Here, we describe the crux of various research articles published by numerous researchers, mostly within the last 10 years to get a better knowledge about the present scenario.

[1]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[2]  Manoj Kumar,et al.  Modified Adomian Decomposition Method and computer implementation for solving singular boundary value problems arising in various physical problems , 2010, Comput. Chem. Eng..

[3]  J. Verwer.  Explicit Runge-Kutta methods for parabolic partial differential equations , 1996 .

[4]  Siwei Luo,et al.  Numerical solution of elliptic partial differential equation using radial basis function neural networks , 2003, Neural Networks.

[5]  Andrew J. Meade,et al.  The numerical solution of linear ordinary differential equations by feedforward neural networks , 1994 .

[6]  Janos Ladik,et al.  Numerical solution of a partial differential equation system describing chemical kinetics and diffusion in a cell with the aid of compartmentalization , 1989 .

[7]  K. V. Roberts,et al.  Automatic production of programmes for solving partial differential equations by finite difference methods , 1972 .

[8]  Isaac E. Lagaris,et al.  Solving differential equations with neural networks: implementation on a DSP platform , 2002, 2002 14th International Conference on Digital Signal Processing Proceedings. DSP 2002 (Cat. No.02TH8628).

[9]  Musa A. Mammadov,et al.  Solving a system of nonlinear integral equations by an RBF network , 2009, Comput. Math. Appl..

[10]  Jincun Liu,et al.  Numerical solutions of the space- and time-fractional coupled Burgers equations by generalized differential transform method , 2011, Appl. Math. Comput..

[11]  T. Tran-Cong,et al.  APPROXIMATION OF FUNCTION AND ITS DERIVATIVES USING RADIAL BASIS FUNCTION NETWORKS , 2003 .

[12]  Y. Shirvany,et al.  Numerical solution of the nonlinear Schrodinger equation by feedforward neural networks , 2008 .

[13]  Parviz Darania,et al.  A method for the numerical solution of the integro-differential equations , 2007, Appl. Math. Comput..

[14]  Manoj Kumar,et al.  Computational Techniques for Solving Differential Equations by Cubic, Quintic, and Sextic Spline , 2009 .

[15]  Hong Chen,et al.  Numerical solution of PDEs via integrated radial basis function networks with adaptive training algorithm , 2011, Appl. Soft Comput..

[16]  Joke Blom,et al.  A comparison of stiff ode solvers for atmospheric chemistry problems , 1995 .

[17]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[18]  Vladimir E. Korepin,et al.  Differential Equations for Quantum Correlation Functions , 1990 .

[19]  Amir Ahmad Dehghani,et al.  Modeling of wheat soaking using two artificial neural networks (MLP and RBF) , 2009 .

[20]  Esmail Babolian,et al.  Numerical solutions of the nonlinear integro-differential equations: Wavelet-Galerkin method and homotopy perturbation method , 2007, Appl. Math. Comput..

[21]  Euripidis Glavas,et al.  Neural network construction and training using grammatical evolution , 2008, Neurocomputing.

[22]  Conor Ryan,et al.  Grammatical Evolution , 2001, Genetic Programming Series.

[23]  R. Ardanuy,et al.  Runge-Kutta methods for numerical solution of stochastic differential equations , 2002 .

[24]  Ahmet Serdar Yilmaz,et al.  Pitch angle control in wind turbines above the rated wind speed by multi-layer perceptron and radial basis function neural networks , 2009, Expert Syst. Appl..

[25]  N. Smaoui,et al.  Modelling the dynamics of nonlinear partial differential equations using neural networks , 2004 .

[26]  P. Zadunaisky.  On the estimation of errors propagated in the numerical integration of ordinary differential equations , 1976 .

[27]  J. Douglas,et al.  On Predictor-Corrector Methods for Nonlinear Parabolic Differential Equations , 1963 .

[28]  S. Ruan,et al.  A delay-differential equation model of HIV infection of CD4(+) T-cells. , 2000, Mathematical biosciences.

[29]  Mohsen Hayati,et al.  Multilayer perceptron neural networks with novel unsupervised training method for numerical solution of the partial differential equations , 2009, Appl. Soft Comput..

[30]  S. Momani,et al.  Solving systems of fractional differential equations using differential transform method , 2008 .

[31]  Fathalla A. Rihan,et al.  Numerical modelling in biosciences using delay differential equations , 2000 .

[32]  V. Thomée.  From finite differences to finite elements a short history of numerical analysis of partial differential equations , 2001 .

[33]  J. Behlke,et al.  A new approximate whole boundary solution of the Lamm differential equation for the analysis of sedimentation velocity experiments. , 2002, Biophysical chemistry.

[34]  A. Kotikov,et al.  Differential equations method: the calculation of vertex-type Feynman diagrams , 1991 .

[35]  A. A. Tseng,et al.  A finite difference scheme with arbitrary mesh systems for solving high-order partial differential equations , 1989 .

[36]  P. E. Zadunaisky.  On the accuracy in the numerical solution of theN-body problem , 1979 .

[37]  Hassan Mohamed El-Hawary,et al.  Spline collocation methods for solving delay-differential equations , 2003, Appl. Math. Comput..

[38]  Ragnar Norberg,et al.  Differential equations for moments of present values in life insurance , 1995 .

[39]  Cristian Filici.  Error estimation in the neural network solution of ordinary differential equations , 2010, Neural Networks.

[40]  Richard Wesley Hamming,et al.  Stable Predictor-Corrector Methods for Ordinary Differential Equations , 1959, JACM.

[42]  Lucie P. Aarts,et al.  Neural Network Method for Solving Partial Differential Equations , 2001, Neural Processing Letters.

[43]  Shital S. Chiddarwar,et al.  Comparison of RBF and MLP neural networks to solve inverse kinematic problem for 6R serial robot by a fusion approach , 2010, Eng. Appl. Artif. Intell..

[44]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[45]  Beny Neta,et al.  Symmetric linear multistep methods for second-order differential equations with periodic solutions , 1985 .

[46]  Kamel Al-khaled,et al.  Numerical solutions for systems of fractional differential equations by the decomposition method , 2005, Appl. Math. Comput..

[47]  S. Sallam,et al.  Numerical solution of general n th-order differential equations via splines , 1990 .

[48]  Alaeddin Malek,et al.  Numerical solution for high order differential equations using a hybrid neural network - Optimization method , 2006, Appl. Math. Comput..

[49]  Zeng Zhe-Zhao,et al.  The Algorithm of Neural Networks on the Initial Value Problems in Ordinary Differential Equations , 2007, IEEE Conference on Industrial Electronics and Applications.

[50]  Andreas Rößler.  Runge-Kutta methods for the numerical solution of stochastic differential equations , 2003 .

[51]  Manoj Kumar,et al.  A boundary value approach for a class of linear singularly perturbed boundary value problems , 2009, Adv. Eng. Softw..

[52]  Manoj Kumar,et al.  A collection of computational techniques for solving singular boundary-value problems , 2009, Adv. Eng. Softw..

[53]  T. Tran-Cong,et al.  Mesh-free radial basis function network methods with domain decomposition for approximation of functions and numerical solution of Poisson's equations , 2002 .

[54]  Khosrow Maleknejad,et al.  Numerical solution of the Fredholm singular integro-differential equation with Cauchy kernel by using Taylor-series expansion and Galerkin method , 2006, Appl. Math. Comput..

[55]  Ju-Hong Lee,et al.  Comparison of generalization ability on solving differential equations using backpropagation and reformulated radial basis function networks , 2009, Neurocomputing.

[56]  Konrad Reif,et al.  Multilayer neural networks for solving a class of partial differential equations , 2000, Neural Networks.

[57]  Bogdan Suchacz,et al.  The recognition of similarities in trace elements content in medicinal plants using MLP and RBF neural networks. , 2006, Talanta.

[58]  Leevan Ling,et al.  A volumetric integral radial basis function method for time-dependent partial differential equations. I. Formulation , 2004 .

[59]  Azim Aminataei,et al.  Numerical solution of Poisson's equation using radial basis function networks on the polar coordinate , 2008, Comput. Math. Appl..

[60]  Prashant Kumar,et al.  Computational method for finding various solutions for a Quasilinear Elliptic Equation of Kirchhoff type , 2009, Adv. Eng. Softw..

[61]  Junwei Lei,et al.  Design of a new kind of RBF neural network based on differential reconstruction , 2005, 2005 International Conference on Neural Networks and Brain.

[62]  Siwei Luo,et al.  Numerical solution of elliptic partial differential equation by growing radial basis function neural networks , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[63]  Mehdi Dehghan,et al.  Numerical solution of the higher-order linear Fredholm integro-differential-difference equation with variable coefficients , 2010, Comput. Math. Appl..

[64]  R. E. White,et al.  One implementation variant of the finite difference method for solving ODEs/DAEs , 2004, Comput. Chem. Eng..

[65]  Nam Mai-Duy,et al.  Solving high order ordinary differential equations with radial basis function networks , 2005 .

[66]  Frank Pettersson,et al.  Method for the selection of inputs and structure of feedforward neural networks , 2006, Comput. Chem. Eng..

[67]  Alaeddin Malek,et al.  Solving initial-boundary value problems for systems of partial differential equations using neural networks and optimization techniques , 2009, J. Frankl. Inst..

[68]  Sohrab Effati,et al.  Artificial neural network approach for solving fuzzy differential equations , 2010, Inf. Sci..

[69]  Euripidis Glavas,et al.  Solving differential equations with constructed neural networks , 2009, Neurocomputing.

[70]  Vivek Dua,et al.  An Artificial Neural Network approximation based decomposition approach for parameter estimation of system of ordinary differential equations , 2011, Comput. Chem. Eng..

[71]  V. Korneev.  Iterative methods of solving systems of equations of the finite element method , 1977 .

[72]  Dimitris G. Papageorgiou,et al.  Neural-network methods for boundary value problems with irregular boundaries , 2000, IEEE Trans. Neural Networks Learn. Syst..

[73]  Manoj Kumar,et al.  Methods for solving singular perturbation problems arising in science and engineering , 2011, Math. Comput. Model..

[74]  Chang Shu,et al.  Integrated radial basis functions‐based differential quadrature method and its performance , 2007 .

[75]  Nam Mai-Duy,et al.  Numerical solution of differential equations using multiquadric radial basis function networks , 2001, Neural Networks.

[76]  Hasan Alli,et al.  The solutions of vibration control problems using artificial neural networks , 2003, J. Frankl. Inst..

[77]  N. Vaziri,et al.  Critical heat flux prediction by using radial basis function and multilayer perceptron neural networks: A comparison study , 2007 .

[78]  Dogan Kaya,et al.  Comparing numerical methods for the solutions of systems of ordinary differential equations , 2004, Appl. Math. Lett..

[79]  T. Tran-Cong,et al.  A Cartesian‐grid collocation method based on radial‐basis‐function networks for solving PDEs in irregular domains , 2007 .

[80]  Luo Siwei,et al.  Numerical solution of differential equations by radial basis function neural networks , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[81]  Manoj Kumar,et al.  Methods for solving singular boundary value problems using splines: a review , 2010 .

[82]  Nam Mai-Duy,et al.  Solving biharmonic problems with scattered-point discretization using indirect radial-basis-function networks , 2006 .

[83]  J. Butcher.  The numerical analysis of ordinary differential equations: Runge-Kutta and general linear methods , 1987 .

[84]  S. Yakowitz,et al.  The Numerical Solution of Ordinary Differential Equations , 1978 .

[85]  Nabil Derbel,et al.  Two coupled neural-networks-based solution of the Hamilton-Jacobi-Bellman equation , 2011, Appl. Soft Comput..

[86]  Sen Bai,et al.  A numerical solution of second-order linear partial differential equations by differential transform , 2006, Appl. Math. Comput..

[87]  Daizhan Cheng,et al.  Numerical solution of damped nonlinear Klein-Gordon equations using variational method and finite element approach , 2005, Appl. Math. Comput..

[88]  Saeid Abbasbandy,et al.  Numerical solution of the system of nonlinear Volterra integro-differential equations with nonlinear differential part by the operational Tau method and error estimation , 2009, J. Comput. Appl. Math..

[89]  A. Golbabai,et al.  Radial basis function networks in the numerical solution of linear integro-differential equations , 2007, Appl. Math. Comput..

[90]  Yan-ze Peng.  Exact solutions for some nonlinear partial differential equations , 2003 .

[91]  Indra Narayan Kar,et al.  Estimation of furnace exit gas temperature (FEGT) using optimized radial basis and back-propagation neural networks , 2008 .