Using Covariance Matrix Adaptation Evolution Strategies for solving different types of differential equations

A novel mesh-free heuristic method for solving differential equations is proposed. The new approach can cope with linear, nonlinear, and partial differential equations (DE), and systems of DEs. Candidate solutions are expressed using a linear combination of kernel functions. Thus, the original problem is transformed into an optimization problem that consists in finding the parameters that define each kernel. The new optimization problem is solved applying a Covariance Matrix Adaptation Evolution Strategy. To increase the accuracy of the results, a Downhill Simplex local search is applied to the best solution found by the mentioned evolutionary algorithm. Our method is applied to 32 differential equations extracted from the literature. All problems are successfully solved, achieving competitive accuracy levels when compared to other heuristic methods. A simple comparison with numerical methods is performed using two partial differential equations to show the pros and cons of the proposed algorithm. To verify the potential of this approach with a more practical problem, an electric circuit is analyzed in depth. The method can obtain the dynamic behavior of the circuit in a parametric way, taking into account different component values.

[1]  Daniel Howard,et al.  Genetic programming of the stochastic interpolation framework: convection–diffusion equation , 2011, Soft Comput..

[2]  Renkuan Guo,et al.  Random fuzzy variable foundation for Grey differential equation modeling , 2008, Soft Comput..

[3]  Johan A. K. Suykens,et al.  Learning solutions to partial differential equations using LS-SVM , 2015, Neurocomputing.

[4]  Daniel A. Ashlock,et al.  Graph-based evolutionary algorithms , 2006, IEEE Transactions on Evolutionary Computation.

[5]  Hadi Sadoghi Yazdi,et al.  Unsupervised adaptive neural-fuzzy inference system for solving differential equations , 2010, Appl. Soft Comput..

[6]  Leif E. Peterson.  Covariance matrix self-adaptation evolution strategies and other metaheuristic techniques for neural adaptive learning , 2011, Soft Comput..

[7]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[8]  E. Süli,et al.  An introduction to numerical analysis , 2003 .

[9]  M. N. Özişik,et al.  Finite Difference Methods in Heat Transfer , 2017 .

[10]  Mohsen Hayati,et al.  Multilayer perceptron neural networks with novel unsupervised training method for numerical solution of the partial differential equations , 2009, Appl. Soft Comput..

[11]  Gavin Brown,et al.  Analytic Solutions to Differential Equations under Graph-Based Genetic Programming , 2010, EuroGP.

[12]  Jonathan R. Bronson,et al.  Adaptive and Unstructured Mesh Cleaving , 2015, Procedia engineering.

[13]  Silvia Ferrari,et al.  A constrained integration (CINT) approach to solving partial differential equations using artificial neural networks , 2015, Neurocomputing.

[14]  Ju-Hong Lee,et al.  Comparison of generalization ability on solving differential equations using backpropagation and reformulated radial basis function networks , 2009, Neurocomputing.

[15]  Konrad Reif,et al.  Multilayer neural networks for solving a class of partial differential equations , 2000, Neural Networks.

[16]  Enrique J. Carmona,et al.  Solving differential equations with Fourier series and Evolution Strategies , 2012, Appl. Soft Comput..

[17]  Isaac E. Lagaris,et al.  Solving differential equations with genetic programming , 2006, Genetic Programming and Evolvable Machines.

[18]  Andy J. Keane,et al.  Genetic Programming Approaches for Solving Elliptic Partial Differential Equations , 2008, IEEE Transactions on Evolutionary Computation.

[19]  Nikolaus Hansen,et al.  Evaluating the CMA Evolution Strategy on Multimodal Test Functions , 2004, PPSN.

[20]  Euripidis Glavas,et al.  Solving differential equations with constructed neural networks , 2009, Neurocomputing.

[21]  Hadi Sadoghi Yazdi,et al.  Unsupervised kernel least mean square algorithm for solving ordinary differential equations , 2011, Neurocomputing.

[22]  Daniel A. Ashlock,et al.  A hybrid genetic programming approach for the analytical solution of differential equations , 2005, Int. J. Gen. Syst..

[23]  Otmar Scherzer,et al.  The CMA-ES on Riemannian Manifolds to Reconstruct Shapes in 3-D Voxel Images , 2010, IEEE Transactions on Evolutionary Computation.

[24]  Bharat K. Soni,et al.  Handbook of Grid Generation , 1998 .

[25]  P. Balasubramaniam,et al.  Solution of matrix Riccati differential equation for nonlinear singular system using genetic programming , 2009, Genetic Programming and Evolvable Machines.

[26]  D. Parisi,et al.  Solving differential equations with unsupervised neural networks , 2003 .

[27]  Tofigh Allahviranloo,et al.  Explicit solutions of fractional differential equations with uncertainty , 2012, Soft Comput..

[28]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[29]  Sujin Bureerat,et al.  Solving Partial Differential Equations Using a New Differential Evolution Algorithm , 2014 .

[30]  Nameer N. El-Emam,et al.  An intelligent computing technique for fluid flow problems using hybrid adaptive neural network and genetic algorithm , 2011, Appl. Soft Comput..

[31]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .

[32]  Mingui Sun,et al.  Solving partial differential equations in real-time using artificial neural network signal processing as an alternative to finite-element analysis , 2003, International Conference on Neural Networks and Signal Processing, 2003. Proceedings of the 2003.

[33]  Maryam Mosleh,et al.  Fuzzy neural network for solving a system of fuzzy differential equations , 2013, Appl. Soft Comput..

[34]  Raja Muhammad Asif Zahoor,et al.  Swarm Intelligence for the Solution of Problems in Differential Equations , 2009, 2009 Second International Conference on Environmental and Computer Science.

[35]  Hong Chen,et al.  Numerical solution of PDEs via integrated radial basis function networks with adaptive training algorithm , 2011, Appl. Soft Comput..

[36]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Tutorial , 2016, ArXiv.

[37]  Xin Yao,et al.  Solving equations by hybrid evolutionary computation techniques , 2000, IEEE Trans. Evol. Comput..

[38]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[39]  Guirong Liu.  Meshfree Methods: Moving Beyond the Finite Element Method, Second Edition , 2009 .

[41]  Amos Ron,et al.  Nonlinear approximation using Gaussian kernels , 2009, 0911.2803.

[42]  R. Tetzlaff,et al.  A learning algorithm for cellular neural networks (CNN) solving nonlinear partial differential equations , 1995, Proceedings of ISSE'95 - International Symposium on Signals, Systems and Electronics.

[43]  Daniel Howard,et al.  Genetic Programming solution of the convection-diffusion equation , 2001 .

[44]  R. LeVeque.  Finite Volume Methods for Hyperbolic Problems: Characteristics and Riemann Problems for Linear Hyperbolic Equations , 2002 .

[45]  Manoj Kumar,et al.  Multilayer perceptrons and radial basis function neural network methods for the solution of differential equations: A survey , 2011, Comput. Math. Appl..

[46]  Kai Yao,et al.  Uncertain differential equation with jumps , 2015, Soft Comput..

[47]  M. Babaei,et al.  A general approach to approximate solutions of nonlinear differential equations using particle swarm optimization , 2013, Appl. Soft Comput..