A constrained integration (CINT) approach to solving partial differential equations using artificial neural networks

This paper presents a novel constrained integration (CINT) method for solving initial boundary value partial differential equations (PDEs). The CINT method combines classical Galerkin methods with a constrained backpropogation training approach to obtain an artificial neural network representation of the PDE solution that approximately satisfies the boundary conditions at every integration step. The advantage of CINT over existing methods is that it is readily applicable to solving PDEs on irregular domains, and requires no special modification for domains with complex geometries. Furthermore, the CINT method provides a semi-analytical solution that is infinitely differentiable. In this paper the CINT method is demonstrated on two hyperbolic and one parabolic initial boundary value problems with a known analytical solutions that can be used for performance comparison. The numerical results show that, when compared to the most efficient finite element methods, the CINT method achieves significant improvements both in terms of computational time and accuracy.

[1]  Mark Ainsworth,et al.  Dispersive and Dissipative Properties of Discontinuous Galerkin Finite Element Methods for the Second-Order Wave Equation , 2006, J. Sci. Comput..

[2]  Donald C. Wunsch,et al.  Clustering with differential evolution particle swarm optimization , 2010, IEEE Congress on Evolutionary Computation.

[3]  David A. Kopriva,et al.  Implementing Spectral Methods for Partial Differential Equations , 2009 .

[4]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[5]  S. Orszag.  Spectral methods for problems in complex geometries , 1980 .

[6]  David A. Kopriva,et al.  Implementing Spectral Methods for Partial Differential Equations: Algorithms for Scientists and Engineers , 2009 .

[7]  Laurens E. Howle,et al.  A comparison of the reduced galerkin and pseudo-spectral methods for simulation of steady Rayleigh-Bénard convection , 1996 .

[8]  Rüdiger Verfürth,et al.  A posteriori error estimates for finite element discretizations of the heat equation , 2003 .

[9]  Sohrab Effati,et al.  Bifurcation analysis of a cellular nonlinear network model via neural network approach , 2014, Neural Computing and Applications.

[10]  Shen Furao,et al.  An incremental network for on-line unsupervised classification and topology learning , 2006, Neural Networks.

[11]  Silvia Ferrari,et al.  A Constrained Optimization Approach to Preserving Prior Knowledge During Incremental Training , 2008, IEEE Transactions on Neural Networks.

[12]  Sven Leyffer,et al.  Nonlinear programming without a penalty function , 2002, Math. Program..

[13]  N. Phan-Thien,et al.  Neural-network-based approximations for solving partial differential equations , 1994 .

[14]  Kevin Stanley McFall,et al.  Artificial Neural Network Method for Solution of Boundary Value Problems With Exact Satisfaction of Arbitrary Boundary Conditions , 2009, IEEE Transactions on Neural Networks.

[15]  Lawrence F. Shampine,et al.  The MATLAB ODE Suite , 1997, SIAM J. Sci. Comput..

[16]  Allan D. Pierce,et al.  Acoustics , 1989 .

[17]  Iliya D. Iliev,et al.  Spectral Methods in Soliton Equations , 1994 .

[18]  Víctor M. Pérez-García,et al.  Spectral Methods for Partial Differential Equations in Irregular Domains: The Spectral Smoothed Boundary Method , 2006, SIAM J. Sci. Comput..

[19]  Alaeddin Malek,et al.  Solving initial-boundary value problems for systems of partial differential equations using neural networks and optimization techniques , 2009, J. Frankl. Inst..

[20]  D. Gottlieb,et al.  Numerical analysis of spectral methods : theory and applications , 1977 .

[21]  Frank L. Lewis,et al.  Fixed-Final-Time-Constrained Optimal Control of Nonlinear Systems Using Neural Network HJB Approach , 2007, IEEE Transactions on Neural Networks.

[22]  Peter M. Pinsky,et al.  Finite element dispersion analysis for the three‐dimensional second‐order scalar wave equation , 1992 .

[23]  J. Boyd.  Large-degree asymptotics and exponential asymptotics for Fourier, Chebyshev and Hermite coefficients and Fourier transforms , 2009 .

[24]  Steven A. Orszag,et al.  CBMS-NSF REGIONAL CONFERENCE SERIES IN APPLIED MATHEMATICS , 1978 .

[25]  Dimitris G. Papageorgiou,et al.  Neural-network methods for boundary value problems with irregular boundaries , 2000, IEEE Trans. Neural Networks Learn. Syst..

[26]  Andrew J. Meade,et al.  Solution of nonlinear ordinary differential equations by feedforward neural networks , 1994 .

[27]  Silvia Ferrari,et al.  A constrained-optimization approach to training neural networks for smooth function approximation and system identification , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[28]  Mohsen Hayati,et al.  Multilayer perceptron neural networks with novel unsupervised training method for numerical solution of the partial differential equations , 2009, Appl. Soft Comput..

[29]  Ganesh K. Venayagamoorthy,et al.  Particle Swarm Optimization with Quantum Infusion for the design of digital filters , 2008, 2008 IEEE Swarm Intelligence Symposium.

[30]  Silvia Ferrari,et al.  A Constrained Backpropagation Approach for the Adaptive Solution of Partial Differential Equations , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[31]  R. Peyret.  Spectral Methods for Incompressible Viscous Flow , 2002 .

[32]  C. Canuto.  Spectral methods in fluid dynamics , 1991 .