Online Adjoint Methods for Optimization of PDEs

We present and mathematically analyze an online adjoint algorithm for the optimization of partial differential equations (PDEs). Traditional adjoint algorithms would typically solve a new adjoint PDE at each optimization iteration, which can be computationally costly. In contrast, an online adjoint algorithm updates the design variables in continuous-time and thus constantly makes progress towards minimizing the objective function. The online adjoint algorithm we consider is similar in spirit to the the pseudotime-stepping, one-shot method which has been previously proposed. Motivated by the application of such methods to engineering problems, we mathematically study the convergence of the online adjoint algorithm. The online adjoint algorithm relies upon a time-relaxed adjoint PDE which provides an estimate of the direction of steepest descent. The algorithm updates this estimate continuously in time, and it asymptotically converges to the exact direction of steepest descent as t Ñ 8. We rigorously prove that the online adjoint algorithm converges to a critical point of the objective function for optimizing the PDE. Under appropriate technical conditions, we also prove a convergence rate for the algorithm. A crucial step in the convergence proof is a multi-scale analysis of the coupled system for the forward PDE, adjoint PDE, and the gradient descent ODE for the design variables.

[1]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[2]  Michael B. Giles,et al.  Adjoint Recovery of Superconvergent Functionals from PDE Approximations , 2000, SIAM Rev..

[3]  Niles A. Pierce,et al.  An Introduction to the Adjoint Approach to Design , 2000 .

[4]  A. Friedman.  Partial Differential Equations of Parabolic Type , 1983 .

[5]  A. Jameson,et al.  Aerodynamic Shape Optimization of Complex Aircraft Configurations via an Adjoint Formulation , 1996 .

[6]  Stefan Ulbrich,et al.  A Continuous Adjoint Approach to Shape Optimization for Navier Stokes Flow , 2009 .

[7]  G. M.,et al.  Partial Differential Equations I , 2023, Applied Mathematical Sciences.

[8]  Subhendu Bikash Hazra,et al.  Direct Treatment of State Constraints in Aerodynamic Shape Optimization Using Simultaneous Pseudo-Time-Stepping , 2007 .

[9]  Volker Schulz,et al.  Simultaneous Pseudo-Timestepping for PDE-Model Based Optimization Problems , 2004, Universität Trier, Mathematik/Informatik, Forschungsbericht.

[10]  A. Jameson,et al.  A COMPARISON OF THE CONTINUOUS AND DISCRETE ADJOINT APPROACH TO AUTOMATIC AERODYNAMIC OPTIMIZATION , 2000 .

[11]  Andreas Griewank,et al.  One-Shot Approaches to Design Optimzation , 2014 .

[12]  S. Taasan.  One shot methods for optimal control of distributed parameter systems 1: Finite dimensional control , 1991 .

[13]  A. Jameson,et al.  Optimum Aerodynamic Design Using the Navier–Stokes Equations , 1997 .

[14]  Antony Jameson,et al.  Aerodynamic Shape Optimization Using the Adjoint Method , 2003 .

[15]  Qiqi Wang,et al.  Simultaneous single-step one-shot optimization with unsteady PDEs , 2015, J. Comput. Appl. Math..

[16]  Hung Tran,et al.  ADJOINT METHODS FOR OBSTACLE PROBLEMS AND WEAKLY COUPLED SYSTEMS OF PDE , 2011, 1103.3226.

[17]  P. Bassanini,et al.  Elliptic Partial Differential Equations of Second Order , 1997 .

[18]  Andreas Griewank,et al.  Automated Extension of Fixed Point PDE Solvers for Optimal Design with Bounded Retardation , 2012, Constrained Optimization and Optimal Control for Partial Differential Equations.

[19]  Nicolas R. Gauger,et al.  One-shot methods in function space for PDE-constrained optimal control problems , 2014, Optim. Methods Softw..

[20]  Stefan Ulbrich,et al.  Optimization with PDE Constraints , 2008, Mathematical modelling.

[21]  Enrique Zuazua,et al.  Continuous adjoint approach for the Spalart−Allmaras model in aerodynamic optimization , 2012 .

[22]  乔花玲,et al.  关于Semigroups of Linear Operators and Applications to Partial Differential Equations的两个注解 , 2003 .

[23]  Stefan Ulbrich,et al.  Convergence of Linearized and Adjoint Approximations for Discontinuous Solutions of Conservation Laws. Part 1: Linearized Approximations and Linearized Output Functionals , 2010, SIAM J. Numer. Anal..

[24]  Konstantinos Spiliopoulos,et al.  Stochastic Gradient Descent in Continuous Time , 2016, SIAM J. Financial Math..

[25]  Bartosz Protas,et al.  Adjoint-based optimization of PDE systems with alternative gradients , 2008, J. Comput. Phys..

[26]  S. Ta'asan.  PSEUDO-TIME METHODS FOR CONSTRAINED OPTIMIZATION PROBLEMS GOVERNED BY PDE , 1995 .

[27]  Konstantinos Spiliopoulos,et al.  PDE-constrained Models with Neural Network Terms: Optimization and Global Convergence , 2021, J. Comput. Phys..

[28]  A. Jameson,et al.  Reduction of the Adjoint Gradient Formula in the Continuous Limit , 2003 .

[29]  Michael B. Giles,et al.  The harmonic adjoint approach to unsteady turbomachinery design , 2002 .

[30]  Damián R. Fernández,et al.  Adjoint method for a tumor growth PDE-constrained optimization problem , 2012, Comput. Math. Appl..

[31]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.

[32]  A. Jameson,et al.  STUDIES OF THE CONTINUOUS AND DISCRETE ADJOINT APPROACHES TO VISCOUS AUTOMATIC AERODYNAMIC SHAPE OPTIMIZATION , 2001 .

[33]  Stefan Ulbrich,et al.  Convergence of Linearized and Adjoint Approximations for Discontinuous Solutions of Conservation Laws. Part 2: Adjoint Approximations and Extensions , 2010, SIAM J. Numer. Anal..