Sensitivity analysis for nonsmooth dynamic systems

Nonsmoothness in dynamic process models can hinder conventional methods for simulation, sensitivity analysis, and optimization, and can be introduced, for example, by transitions in flow regime or thermodynamic phase, or through discrete changes in the operating mode of a process. While dedicated numerical methods exist for nonsmooth problems, these methods require generalized derivative information that can be difficult to furnish. This thesis presents some of the first automatable methods for computing these generalized derivatives. Firstly, Nesterov's lexicographic derivatives are shown to be elements of the plenary hull of Clarke's generalized Jacobian whenever they exist. Lexicographic derivatives thus provide useful local sensitivity information for use in numerical methods for nonsmooth problems. A vector forward mode of automatic differentiation is developed and implemented to evaluate lexicographic derivatives for finite compositions of simple lexicographically smooth functions, including the standard arithmetic operations, trigonometric functions, exp / log, piecewise differentiable functions such as the absolute-value function, and other nonsmooth functions such as the Euclidean norm. This method is accurate, automatable, and computationally inexpensive. Next, given a parametric ordinary differential equation (ODE) with a lexicographically smooth right-hand side function, parametric lexicographic derivatives of a solution trajectory are described in terms of the unique solution of a certain auxiliary ODE. A numerical method is developed and implemented to solve this auxiliary ODE, when the right-hand side function for the original ODE is a composition of absolute-value functions and analytic functions. Computationally tractable sufficient conditions are also presented for differentiability of the original ODE solution with respect to system parameters. Sufficient conditions are developed under which local inverse and implicit functions are lexicographically smooth. These conditions are combined with the results above to describe parametric lexicographic derivatives for certain hybrid discrete/continuous systems, including some systems whose discrete mode trajectories change when parameters are perturbed.

[1]  Lorenz T. Biegler,et al.  MPEC problem formulations and solution strategies with chemical engineering applications , 2008, Comput. Chem. Eng..

[2]  Jean-Baptiste Hiriart-Urruty,et al.  Characterizations of the plenary hull of the generalized Jacobian matrix , 1982 .

[3]  Donald E. Kirk,et al.  Optimal control theory : an introduction , 1970 .

[4]  D. Klatte Nonsmooth equations in optimization , 2002 .

[5]  Ignacio E. Grossmann,et al.  Simultaneous optimization and heat integration of chemical processes , 1986 .

[6]  Michael Ulbrich,et al.  Semismooth Newton Methods for Variational Inequalities and Constrained Optimization Problems in Function Spaces , 2011, MOS-SIAM Series on Optimization.

[7]  M. Kanat Camlibel,et al.  Continuous Piecewise Affine Dynamical Systems do not Exhibit Zeno Behavior , 2011, IEEE Transactions on Automatic Control.

[8]  D. Bertsekas Nondifferentiable optimization via approximation , 1975 .

[9]  T. H. III Sweetser,et al.  A minimal set-valued strong derivative for vector-valued Lipschitz functions , 1977 .

[10]  S. Sastry,et al.  On the existence of executions of hybrid automata , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).

[11]  Spencer D. Schaber Tools for dynamic model development , 2014 .

[12]  Georg Stadler,et al.  Elliptic optimal control problems with L1-control cost and applications for the placement of control devices , 2009, Comput. Optim. Appl..

[13]  Michael C. Ferris,et al.  Interfaces to PATH 3.0: Design, Implementation and Usage , 1999, Comput. Optim. Appl..

[14]  Jong-Shi Pang,et al.  Differential variational inequalities , 2008, Math. Program..

[15]  Paul I. Barton,et al.  State event location in differential-algebraic models , 1996, TOMC.

[16]  François E. Cellier Combined continuous/discrete system simulation by use of digital computers , 1979 .

[17]  Yurii Nesterov,et al.  Lexicographic differentiation of nonsmooth functions , 2005, Math. Program..

[18]  Arthur W. Westerberg,et al.  Acceleration of cyclic steady state convergence for pressure swing adsorption models , 1992 .

[19]  Paul I. Barton,et al.  Generalized Derivatives for Hybrid Systems , 2017, IEEE Transactions on Automatic Control.

[20]  Daniel Scholz,et al.  Theoretical rate of convergence for interval inclusion functions , 2012, J. Glob. Optim..

[21]  Daniel Ralph,et al.  Sensitivity analysis of composite piecewise smooth equations , 1997, Math. Program..

[22]  Michael Hintermüller,et al.  A SQP-Semismooth Newton-type Algorithm applied to Control of the instationary Navier--Stokes System Subject to Control Constraints , 2006, SIAM J. Optim..

[23]  S. Dirkse,et al.  The path solver: a nommonotone stabilization scheme for mixed complementarity problems , 1995 .

[24]  R. Horst,et al.  Global Optimization: Deterministic Approaches , 1992 .

[25]  Aleksej F. Filippov,et al.  Differential Equations with Discontinuous Righthand Sides , 1988, Mathematics and Its Applications.

[26]  D. Willett,et al.  On the discrete analogues of some generalizations of Gronwall's inequality , 1965 .

[27]  R. Baker Kearfott,et al.  The cluster problem in multivariate global optimization , 1994, J. Glob. Optim..

[28]  Jong-Shi Pang,et al.  NE/SQP: A robust algorithm for the nonlinear complementarity problem , 1993, Math. Program..

[29]  G. Alefeld,et al.  Interval analysis: theory and applications , 2000 .

[30]  Jong-Shi Pang,et al.  Piecewise Smoothness, Local Invertibility, and Parametric Analysis of Normal Maps , 1996, Math. Oper. Res..

[31]  Jirí V. Outrata,et al.  Shape Optimization in Three-Dimensional Contact Problems with Coulomb Friction , 2009, SIAM J. Optim..

[32]  Paul I. Barton,et al.  Evaluating an element of the Clarke generalized Jacobian of a composite piecewise differentiable function , 2013, TOMS.

[33]  C. Lemaréchal,et al.  ON A BUNDLE ALGORITHM FOR NONSMOOTH OPTIMIZATION , 1981 .

[34]  J. J. Moré,et al.  Smoothing of mixed complementarity problems , 1995 .

[35]  Paul I. Barton,et al.  Nonlinear convex and concave relaxations for the solutions of parametric ODEs , 2013 .

[36]  Paul I. Barton,et al.  Convex and Concave Relaxations for the Parametric Solutions of Semi-explicit Index-One Differential-Algebraic Equations , 2012, Journal of Optimization Theory and Applications.

[37]  Cyril Imbert,et al.  Support functions of the Clarke generalized Jacobian and of its plenary hull , 2002 .

[38]  Liqun Qi,et al.  A nonsmooth version of Newton's method , 1993, Math. Program..

[39]  Zdzisław Denkowski,et al.  Set-Valued Analysis , 2021 .

[40]  Héctor J. Sussmann Bounds on the number of switchings for trajectories of piecewise analytic vector fields , 1982 .

[41]  A. Neumaier,et al.  A global optimization method, αBB, for general twice-differentiable constrained NLPs — I. Theoretical advances , 1998 .

[42]  A. Griewank Automatic Directional Differentiation of Nonsmooth Composite Functions , 1995 .

[43]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[44]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[45]  Paul I. Barton,et al.  A vector forward mode of automatic differentiation for generalized derivative evaluation , 2015, Optim. Methods Softw..

[46]  Alexander Mitsos,et al.  Multivariate McCormick relaxations , 2014, J. Glob. Optim..

[47]  K. Kiwiel Methods of Descent for Nondifferentiable Optimization , 1985 .

[48]  Jong-Shi Pang,et al.  Strongly Regular Differential Variational Systems , 2007, IEEE Transactions on Automatic Control.

[49]  Le Quang Thuan Non-Zenoness of piecewise affine dynamical systems and affine complementarity systems with inputs , 2014 .

[50]  Paul I. Barton,et al.  Bounds on reachable sets using ordinary differential equations with linear programs embedded , 2016, IMA J. Math. Control. Inf..

[51]  Matthew D. Stuber,et al.  Convex and concave relaxations of implicit functions , 2015, Optim. Methods Softw..

[52]  Garth P. McCormick,et al.  Computability of global solutions to factorable nonconvex programs: Part I — Convex underestimating problems , 1976, Math. Program..

[53]  José Herskovits,et al.  Globally Convergent Cutting Plane Method for Nonconvex Nonsmooth Minimization , 2011, J. Optim. Theory Appl..

[54]  J. M. Schumacher,et al.  Complementarity systems in optimization , 2004, Math. Program..

[55]  Andreas Griewank,et al.  On stable piecewise linearization and generalized algorithmic differentiation , 2013, Optim. Methods Softw..

[56]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[57]  H. Whitney Analytic Extensions of Differentiable Functions Defined in Closed Sets , 1934 .

[58]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[59]  Joseph K. Scott Reachability analysis and deterministic global optimization of differential-algebraic systems , 2012 .

[60]  N. G. Parke,et al.  Ordinary Differential Equations. , 1958 .

[61]  L. Petzold,et al.  Numerical methods and software for sensitivity analysis of differential-algebraic systems , 1986 .

[62]  U. Naumann,et al.  Adjoint Mode Computation of Subgradients for McCormick Relaxations , 2012 .

[63]  P I Barton,et al.  A reliable simulator for dynamic flux balance analysis , 2013, Biotechnology and bioengineering.

[64]  M. Egerstedt,et al.  On the regularization of Zeno hybrid automata , 1999 .

[65]  Paul I. Barton,et al.  Generalized gradient elements for nonsmooth optimal control problems , 2014, 53rd IEEE Conference on Decision and Control.

[66]  P. I. Barton,et al.  Outer approximation algorithms for separable nonconvex mixed-integer nonlinear programs , 2004, Math. Program..

[67]  F. Clarke Optimization And Nonsmooth Analysis , 1983 .

[68]  A. Neumaier Interval methods for systems of equations , 1990 .

[69]  Xiang Li,et al.  Nonconvex Generalized Benders Decomposition for Stochastic Separable Mixed-Integer Nonlinear Programs , 2011, J. Optim. Theory Appl..

[70]  Leo Liberti,et al.  Convex Envelopes of Monomials of Odd Degree , 2003, J. Glob. Optim..

[71]  Alexander Mitsos,et al.  Convergence rate of McCormick relaxations , 2012, J. Glob. Optim..

[72]  Paul I. Barton,et al.  McCormick-Based Relaxations of Algorithms , 2009, SIAM J. Optim..

[73]  P. I. Barton,et al.  Parametric sensitivity functions for hybrid discrete/continuous systems , 1999 .

[74]  Jan Vlcek,et al.  A bundle-Newton method for nonsmooth unconstrained minimization , 1998, Math. Program..

[75]  Vera Zeidan,et al.  Infinite dimensional clarke generalized Jacobian , 2007 .

[76]  Paul I. Barton,et al.  Generalized Derivatives for Solutions of Parametric Ordinary Differential Equations with Non-differentiable Right-Hand Sides , 2014, J. Optim. Theory Appl..

[77]  M. Kojima,et al.  EXTENSION OF NEWTON AND QUASI-NEWTON METHODS TO SYSTEMS OF PC^1 EQUATIONS , 1986 .

[78]  Jiří V. Outrata,et al.  Shape Optimization in 2D Contact Problems with Given Friction and a Solution-Dependent Coefficient of Friction , 2012 .

[79]  P. I. Barton,et al.  DAEPACK: An Open Modeling Environment for Legacy Models , 2000 .

[80]  Paul I. Barton,et al.  The cluster problem revisited , 2013, Journal of Global Optimization.

[81]  A. Griewank,et al.  On the smoothness of convex envelopes , 1990 .

[82]  Shengtai Li,et al.  Adjoint Sensitivity Analysis for Differential-Algebraic Equations: The Adjoint DAE System and Its Numerical Solution , 2002, SIAM J. Sci. Comput..

[83]  Bangti Jin,et al.  A Semismooth Newton Method for L1 Data Fitting with Automatic Choice of Regularization Parameters and Noise Calibration , 2010, SIAM J. Imaging Sci..

[84]  M. A. Jenkins,et al.  A Three-Stage Algorithm for Real Polynomials Using Quadratic Iteration , 1970 .

[85]  Jan Vlcek,et al.  Algorithm 811: NDA: algorithms for nondifferentiable optimization , 2001, TOMS.

[86]  Paul I. Barton,et al.  Reverse propagation of McCormick relaxations , 2015, Journal of Global Optimization.

[87]  Bernard Friedland,et al.  Control System Design: An Introduction to State-Space Methods , 1987 .

[88]  John N. Tsitsiklis,et al.  Introduction to linear optimization , 1997, Athena scientific optimization and computation series.

[89]  M A Henson,et al.  Steady-state and dynamic flux balance analysis of ethanol production by Saccharomyces cerevisiae. , 2009, IET systems biology.

[90]  Francisco Facchinei,et al.  An LP-Newton method: nonsmooth equations, KKT systems, and nonisolated solutions , 2013, Mathematical Programming.

[91]  Harold R. Parks,et al.  A Primer of Real Analytic Functions , 1992 .

[92]  R. Mifflin Semismooth and Semiconvex Functions in Constrained Optimization , 1977 .

[93]  Ignacio E. Grossmann,et al.  Modeling multistream heat exchangers with and without phase changes for simultaneous optimization and heat integration , 2012 .

[94]  W. Rudin Principles of mathematical analysis , 1964 .

[95]  Ramon E. Moore Methods and applications of interval analysis , 1979, SIAM studies in applied mathematics.

[96]  Mehmet Yunt,et al.  Nonsmooth dynamic optimization of systems with varying structure , 2011 .

[97]  P. I. Barton,et al.  Efficient sensitivity analysis of large-scale differential-algebraic systems , 1997 .

[98]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[99]  Kazufumi Ito,et al.  The Primal-Dual Active Set Strategy as a Semismooth Newton Method , 2002, SIAM J. Optim..

[100]  Arjan van der Schaft,et al.  The complementary-slackness class of hybrid systems , 1996, Math. Control. Signals Syst..

[101]  Brahim Benyahia,et al.  A Plant-Wide Dynamic Model of a Continuous Pharmaceutical Process , 2012 .

[102]  Huifu Xu,et al.  Set-valued approximations and Newton’s methods , 1999, Math. Program..

[103]  Liqun Qi,et al.  Convergence Analysis of Some Algorithms for Solving Nonsmooth Equations , 1993, Math. Oper. Res..

[104]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[105]  Jong-Shi Pang,et al.  Solution dependence on initial conditions in differential variational inequalities , 2008, Math. Program..

[106]  E. Coddington,et al.  Theory of Ordinary Differential Equations , 1955 .

[107]  Vera Zeidan,et al.  Infinite dimensional generalized Jacobian: Properties and calculus rules , 2008 .

[108]  Achim Wechsung,et al.  Global optimization in reduced space , 2014 .

[109]  Daniel Ralph,et al.  Global Convergence of Damped Newton's Method for Nonsmooth Equations via the Path Search , 1994, Math. Oper. Res..

[110]  Defeng Sun,et al.  Smoothing Functions and Smoothing Newton Method for Complementarity and Variational Inequality Problems , 2002 .

[111]  P. I. Barton,et al.  Dynamic optimization of hybrid systems , 1998 .

[112]  L. Biegler,et al.  Nonlinear Programming Superstructure for Optimal Dynamic Operations of Simulated Moving Bed Processes , 2006 .

[113]  Paul I. Barton,et al.  Improved relaxations for the parametric solutions of ODEs using differential inequalities , 2012, Journal of Global Optimization.