Discrete second order adjoints in atmospheric chemical transport modeling

Atmospheric chemical transport models (CTMs) are essential tools for the study of air pollution, for environmental policy decisions, for the interpretation of observational data, and for producing air quality forecasts. Many air quality studies require sensitivity analyses, i.e., the computation of derivatives of the model output with respect to model parameters. The derivatives of a cost functional (defined on the model output) with respect to a large number of model parameters can be calculated efficiently through adjoint sensitivity analysis. While the traditional (first order) adjoint models give the gradient of the cost functional with respect to parameters, second order adjoint models give second derivative information in the form of products between the Hessian of the cost functional and a vector (representing a perturbation in sensitivity analysis, a search direction in optimization, an eigenvector, etc.). In this paper we discuss the mathematical foundations of the discrete second order adjoint sensitivity method and present a complete set of computational tools for performing second order sensitivity studies in three-dimensional atmospheric CTMs. The tools include discrete second order adjoints of Runge-Kutta and of Rosenbrock time stepping methods for stiff equations together with efficient implementation strategies. Numerical examples illustrate the use of these computational tools in important applications like sensitivity analysis, optimization, uncertainty quantification and the calculation of directions of maximal error growth in three-dimensional atmospheric CTMs.

[1]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[2]  J. Blom,et al.  Benchmarking Stiff ODE Solvers for Atmospheric Chemistry Problems II: Rosenbrock Methods , 1996 .

[3]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[4]  M. Berz,et al.  Efficient high-order methods for ODEs and DAEs , 2000 .

[5]  Adrian Sandu,et al.  Chemical data assimilation of Transport and Chemical Evolution over the Pacific (TRACE-P) aircraft measurements , 2006 .

[6]  E. Hairer,et al.  Stiff and differential-algebraic problems , 1991 .

[7]  J. Daniel.  The Conjugate Gradient Method for Linear and Nonlinear Operator Equations , 1967 .

[8]  O. Talagrand,et al.  4D-variational data assimilation with an adjoint air quality model for emission analysis , 2000, Environ. Model. Softw..

[9]  Jorge Nocedal,et al.  Large Scale Unconstrained Optimization , 1997 .

[10]  Harley Flanders.  Application of AD to a family of periodic functions , 2000 .

[11]  J. Verwer,et al.  Numerical solution of time-dependent advection-diffusion-reaction equations , 2003 .

[12]  Philip B. Russell,et al.  International Consortium for Atmospheric Research on Transport and Transformation (ICARTT): North America to Europe—Overview of the 2004 summer field study , 2006 .

[13]  I. M. Navonb,et al.  Comparison of advanced large-scale minimization algorithms for the solution of inverse problems , 2004 .

[14]  I. Michael Navon,et al.  The analysis of an ill-posed problem using multi-scale resolution and second-order adjoint techniques , 2001 .

[15]  L. K. Peters,et al.  A second generation model for regional-scale transport/chemistry/deposition , 1986 .

[16]  E. Hairer,et al.  Solving Ordinary Differential Equations II: Stiff and Differential-Algebraic Problems , 2010 .

[17]  Adrian Sandu,et al.  A response spectral decomposition approach for the control of nonlinear mechanical systems , 2008 .

[18]  J. Verwer,et al.  Analysis of operator splitting for advection-diffusion-reaction problems from air pollution modelling , 1999 .

[19]  Emil M. Constantinescu,et al.  Predicting air quality: Improvements through advanced methods to integrate models and measurements , 2008, J. Comput. Phys..

[20]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[21]  P. Courtier,et al.  Variational Assimilation of Meteorological Observations With the Adjoint Vorticity Equation. I: Theory , 2007 .

[22]  SanduAdrian,et al.  Predicting air quality , 2008 .

[23]  Adrian Sandu,et al.  Benchmarking stiff ode solvers for atmospheric chemistry problems II: Rosenbrock solvers , 1997 .

[24]  A. Walther,et al.  Towards Matrix‐Free AD‐Based Preconditioning of KKT Systems in PDE‐Constrained Optimization , 2005 .

[25]  J. Barkmeijer,et al.  Singular vectors and estimates of the analysis‐error covariance metric , 1998 .

[26]  Stéphane Garreau,et al.  Application of higher order derivatives to parameterization , 2000 .

[27]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[28]  F. L. Dimet,et al.  Variational algorithms for analysis and assimilation of meteorological observations: theoretical aspects , 1986 .

[29]  B. Sportisse.  An Analysis of Operator Splitting Techniques in the Stiff Case , 2000 .

[30]  J. Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[31]  David J. Lary,et al.  Lagrangian four‐dimensional variational data assimilation of chemical species , 1995 .

[32]  Ernst Hairer,et al.  Solving Ordinary Differential Equations I: Nonstiff Problems , 2009 .

[33]  David G. Streets,et al.  Regional‐scale chemical transport modeling in support of the analysis of observations obtained during the TRACE‐P experiment , 2003 .

[34]  Andreas Griewank,et al.  Automatic Differentiation of Algorithms: From Simulation to Optimization , 2000, Springer New York.

[35]  Jorge Nocedal,et al.  Enriched Methods for Large-Scale Unconstrained Optimization , 2002, Comput. Optim. Appl..

[36]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[37]  Jacques Verron,et al.  Sensitivity Analysis in Variational Data Assimilation (gtSpecial IssueltData Assimilation in Meteology and Oceanography: Theory and Practice) , 1997 .

[38]  Aleksey K. Alekseev,et al.  Comparison of advanced large-scale minimization algorithms for the solution of inverse ill-posed problems , 2009, Optim. Methods Softw..

[39]  William W. Hager,et al.  Runge-Kutta methods in optimal control and the transformed adjoint system , 2000, Numerische Mathematik.

[40]  Derya B. Özyurt,et al.  Cheap Second Order Directional Derivatives of Stiff ODE Embedded Functionals , 2005, SIAM J. Sci. Comput..

[41]  Tim N. Palmer,et al.  Potential improvement to forecasts of two severe storms using targeted observations , 2002 .

[42]  S. Lakshmivarahan,et al.  Second-order approximation to the 3DVAR cost function: application to analysis/forecast , 2003 .

[43]  Dacian N. Daescu,et al.  Adaptive observations in the context of 4D-Var data assimilation , 2004 .

[44]  H. V. D. Vorst,et al.  Jacobi-davidson type methods for generalized eigenproblems and polynomial eigenproblems , 1995 .

[45]  Jassim A. Al-Saadi,et al.  Influence of lateral and top boundary conditions on regional air quality prediction: A multiscale study coupling regional and global chemical transport models , 2007 .

[46]  V. I. Zubov,et al.  FAD method to compute second order derivatives , 2000 .

[47]  Ionel M. Navon,et al.  An Analysis of a Hybrid Optimization Method for Variational Data Assimilation , 2003 .

[48]  I. Michael Navon,et al.  On Estimation of Temperature Uncertainty Using the Second Order Adjoint Problem , 2002 .

[49]  G. Carmichael,et al.  Direct and adjoint sensitivity analysis of chemical kinetic systems with KPP: II—numerical validation and applications , 2003 .

[50]  David A. H. Jacobs,et al.  The State of the Art in Numerical Analysis. , 1978 .

[51]  Adrian Sandu,et al.  Total Energy Singular Vectors for Atmospheric Chemical Transport Models , 2005, International Conference on Computational Science.

[52]  Florian A. Potra,et al.  The kinetic preprocessor KPP*/a software environment for solving chemical kinetics , 2002 .

[53]  F. Krogh,et al.  Solving Ordinary Differential Equations , 2019, Programming for Computations - Python.

[54]  Adrian Sandu,et al.  On the properties of discrete adjoints of numerical methods for the advection equation , 2008 .

[55]  A. Simmons,et al.  The ECMWF operational implementation of four‐dimensional variational assimilation. I: Experimental results with simplified physics , 2007 .

[56]  P. Courtier,et al.  A strategy for operational implementation of 4D‐Var, using an incremental approach , 1994 .

[57]  Zhi Wang,et al.  The second order adjoint analysis: Theory and applications , 1992 .

[58]  Paul I. Barton,et al.  Application of Targeted Automatic Differentiation to Large-Scale Dynamic Optimization , 2006 .

[59]  James W. Daniel.  A Correction Concerning the Convergence Rate for the Conjugate Gradient Method , 1970 .

[60]  William P L Carter,et al.  Documentation of the SAPRC-99 chemical mechanism for VOC reactivity assessment. Volume 2. , 2000 .

[61]  J. Daniel.  Convergence of the conjugate gradient method with computationally convenient modifications , 1967 .

[62]  Adrian Sandu,et al.  Adjoint sensitivity analysis of regional air quality models , 2005 .

[63]  Zhi Wang,et al.  The Adjoint Newton Algorithm for Large-Scale Unconstrained Optimization in Meteorology Applications , 1998, Comput. Optim. Appl..

[64]  Adrian Sandu,et al.  Adjoint sensitivity analysis of ozone nonattainment over the continental United States. , 2006, Environmental science & technology.

[65]  E. Hairer,et al.  Solving Ordinary Differential Equations I , 1987 .

[66]  Emil M. Constantinescu,et al.  Autoregressive models of background errors for chemical data assimilation , 2007 .

[67]  Paul I. Barton,et al.  Large-Scale Dynamic Optimization Using the Directional Second-Order Adjoint Method , 2005 .

[68]  G. Carmichael,et al.  Direct and adjoint sensitivity analysis of chemical kinetic systems with KPP: Part I—theory and software tools , 2003, Atmospheric Environment.

[69]  Ionel M. Navon,et al.  Efficiency of a POD-based reduced second-order adjoint model in 4 D-Var data assimilation , 2006 .

[70]  Hendrik Elbern,et al.  Ozone episode analysis by four-dimensional variational chemistry data assimilation , 2001 .

[71]  Jacques Verron,et al.  Sensitivity Analysis in Variational Data Assimilation , 1997 .

[72]  Roberto Buizza,et al.  3D‐Var Hessian singular vectors and their potential use in the ECMWF ensemble prediction system , 1999 .

[73]  Isabelle Charpentier,et al.  Second order exact derivatives to perform optimization on self-consistent integral equations problems , 2000 .

[74]  Roberto Buizza,et al.  Tropical singular vectors computed with linearized diabatic physics , 2001 .

[75]  Ionel M. Navon,et al.  Second-Order Information in Data Assimilation* , 2002 .

[76]  José Luis Morales,et al.  A numerical study of limited memory BFGS methods , 2002, Appl. Math. Lett..

[77]  Arjo Segers,et al.  Data Assimilation in Atmospheric Chemistry Models Using Kalman Filtering , 2002 .

[78]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[79]  C. Tomlin,et al.  Second order adjoint-based optimization of ordinary and partial differential equations with application to air traffic flow , 2005, Proceedings of the 2005, American Control Conference, 2005..

[80]  Andrew C. Lorenc,et al.  Analysis methods for numerical weather prediction , 1986 .

[81]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..