A Constrained Backpropagation Approach for the Adaptive Solution of Partial Differential Equations

This paper presents a constrained backpropagation (CPROP) methodology for solving nonlinear elliptic and parabolic partial differential equations (PDEs) adaptively, subject to changes in the PDE parameters or external forcing. Unlike existing methods based on penalty functions or Lagrange multipliers, CPROP solves the constrained optimization problem associated with training a neural network to approximate the PDE solution by means of direct elimination. As a result, CPROP reduces the dimensionality of the optimization problem, while satisfying the equality constraints associated with the boundary and initial conditions exactly, at every iteration of the algorithm. The effectiveness of this method is demonstrated through several examples, including nonlinear elliptic and parabolic PDEs with changing parameters and nonhomogeneous terms.

[1]  N. Phan-Thien,et al.  Neural-network-based approximations for solving partial differential equations , 1994 .

[2]  Silvia Ferrari,et al.  A constrained-optimization approach to training neural networks for smooth function approximation and system identification , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[3]  Mohsen Hayati,et al.  Multilayer perceptron neural networks with novel unsupervised training method for numerical solution of the partial differential equations , 2009, Appl. Soft Comput..

[4]  Alaeddin Malek,et al.  Solving initial-boundary value problems for systems of partial differential equations using neural networks and optimization techniques , 2009, J. Frankl. Inst..

[5]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[6]  Silvia Ferrari,et al.  A Constrained Optimization Approach to Preserving Prior Knowledge During Incremental Training , 2008, IEEE Transactions on Neural Networks.

[7]  L. L. Rogers,et al.  Optimization of groundwater remediation using artificial neural networks with parallel solute transport modeling , 1994 .

[8]  Giuseppe Fedele.  A new method to estimate a first-order plus time delay model from step response , 2009, J. Frankl. Inst..

[9]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[10]  Pratibha Mishra,et al.  Advanced Engineering Mathematics , 2013 .

[11]  Gilles Pagès,et al.  Approximations of Functions by a Multilayer Perceptron: a New Approach , 1997, Neural Networks.

[12]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[13]  G. Smith,et al.  Numerical Solution of Partial Differential Equations: Finite Difference Methods , 1978 .

[14]  P. Werbos.  Backwards Differentiation in AD and Neural Nets: Past Links and New Opportunities , 2006 .

[15]  Jin H. Huang,et al.  Detection of cracks using neural networks and computational mechanics , 2002 .

[16]  Dimitris G. Papageorgiou,et al.  Neural-network methods for boundary value problems with irregular boundaries , 2000, IEEE Trans. Neural Networks Learn. Syst..

[17]  Yoh-Han Pao,et al.  Stochastic choice of basis functions in adaptive function approximation and the functional-link net , 1995, IEEE Trans. Neural Networks.

[18]  Frank L. Lewis,et al.  Fixed-Final-Time-Constrained Optimal Control of Nonlinear Systems Using Neural Network HJB Approach , 2007, IEEE Transactions on Neural Networks.

[19]  J. E. Glynn,et al.  Numerical Recipes: The Art of Scientific Computing , 1989 .

[20]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[21]  F. W. Kellaway,et al.  Advanced Engineering Mathematics , 1969, The Mathematical Gazette.

[22]  Robert F. Stengel,et al.  Optimal Control and Estimation , 1994 .

[23]  Kurt Hornik,et al.  Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks , 1990, Neural Networks.

[24]  Robert F. Stengel,et al.  Smooth function approximation using neural networks , 2005, IEEE Transactions on Neural Networks.

[25]  Xin Li,et al.  Simultaneous approximations of multivariate functions and their derivatives by neural networks with one hidden layer , 1996, Neurocomputing.

[26]  Long Chen.  FINITE ELEMENT METHOD , 2013 .

[27]  G. Bower,et al.  THE PSYCHOLOGY OF LEARNING AND M·OTIVATION , 2001 .

[28]  Kevin Stanley McFall,et al.  Artificial Neural Network Method for Solution of Boundary Value Problems With Exact Satisfaction of Arbitrary Boundary Conditions , 2009, IEEE Transactions on Neural Networks.

[29]  Michael Hüsken,et al.  Fast Adaptation of the Solution of Differential Equations to Changing Constraints , 2000 .

[30]  Caesar Saloma,et al.  Computational complexity and the observation of physical signals , 1993 .