Nonlinear robust optimization via sequential convex bilevel programming

Abstract In this paper, we present a novel sequential convex bilevel programming algorithm for the numerical solution of structured nonlinear min–max problems which arise in the context of semi-infinite programming. Here, our main motivation are nonlinear inequality constrained robust optimization problems. In the first part of the paper, we propose a conservative approximation strategy for such nonlinear and non-convex robust optimization problems: under the assumption that an upper bound for the curvature of the inequality constraints with respect to the uncertainty is given, we show how to formulate a lower-level concave min–max problem which approximates the robust counterpart in a conservative way. This approximation turns out to be exact in some relevant special cases and can be proven to be less conservative than existing approximation techniques that are based on linearization with respect to the uncertainties. In the second part of the paper, we review existing theory on optimality conditions for nonlinear lower-level concave min–max problems which arise in the context of semi-infinite programming. Regarding the optimality conditions for the concave lower level maximization problems as a constraint of the upper level minimization problem, we end up with a structured mathematical program with complementarity constraints (MPCC). The special hierarchical structure of this MPCC can be exploited in a novel sequential convex bilevel programming algorithm. We discuss the surprisingly strong global and locally quadratic convergence properties of this method, which can in this form neither be obtained with existing SQP methods nor with interior point relaxation techniques for general MPCCs. Finally, we discuss the application fields and implementation details of the new method and demonstrate the performance with a numerical example.

[1]  Danny C. Sorensen,et al.  On the use of directions of negative curvature in a modified newton method , 1979, Math. Program..

[2]  C. Lemaréchal,et al.  The watchdog technique for forcing convergence in algorithms for constrained optimization , 1982 .

[3]  S. M. Robinson First Order Conditions for General Nonlinear Optimization , 1976 .

[4]  Shih-Ping Han A globally convergent method for nonlinear programming , 1975 .

[5]  Arkadi Nemirovski,et al.  Robust Truss Topology Design via Semidefinite Programming , 1997, SIAM J. Optim..

[6]  M. Anitescu NONLINEAR PROGRAMS WITH UNBOUNDED LAGRANGE MULTIPLIER SETS , 2000 .

[7]  G. Still,et al.  Second order optimality conditions for generalized semi-infinite programming problems , 1995 .

[8]  Arkadi Nemirovski,et al.  Robust Convex Optimization , 1998, Math. Oper. Res..

[9]  J. Douglas Faires,et al.  Numerical Analysis , 1981 .

[10]  Stephen P. Boyd,et al.  Extending Scope of Robust Optimization: Comprehensive Robust Counterparts of Uncertain Problems , 2006, Math. Program..

[11]  Paul I. Barton,et al.  Global solution of semi-infinite programs , 2004 .

[12]  Georg J. Still,et al.  Generalized semi-infinite programming: numerical aspects , 2001 .

[13]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[14]  George McMurdo An Analysis of UK MARC exchange tape variable data field subfield lengths, 1950–1980 , 1982 .

[15]  Christodoulos A. Floudas,et al.  Deterministic global optimization - theory, methods and applications , 2010, Nonconvex optimization and its applications.

[16]  Defeng Sun,et al.  Second-Order Algorithms for Generalized Finite and Semi-Infinite Min-Max Problems , 2001, SIAM J. Optim..

[17]  Andrew R. Conn,et al.  Nonlinear programming via an exact penalty function: Asymptotic analysis , 1982, Math. Program..

[18]  L El Ghaoui,et al.  ROBUST SOLUTIONS TO LEAST-SQUARE PROBLEMS TO UNCERTAIN DATA MATRICES , 1997 .

[19]  Jianzhon Zhang,et al.  A new branch and bound algorithm for solving quadratic programs with linear complementarity constraints , 2002 .

[20]  Siddhartha Mukhopadhyay,et al.  An extended finitely recursive process model for discrete event systems , 1995, IEEE Transactions on Systems, Man, and Cybernetics.

[21]  E. Polak,et al.  A recursive quadratic programming algorithm for semi-infinite optimization problems , 1982 .

[22]  Moritz Diehl,et al.  Optimal control for power generating kites , 2007, 2007 European Control Conference (ECC).

[23]  Kees Roos,et al.  Robust Solutions of Uncertain Quadratic and Conic-Quadratic Problems , 2002, SIAM J. Optim..

[24]  Oliver Stein,et al.  On Linear and Linearized Generalized Semi-Infinite Optimization Problems , 2001, Ann. Oper. Res..

[25]  Defeng Sun,et al.  First-Order Algorithms for Generalized Semi-Infinite Min-Max Problems , 1999, Comput. Optim. Appl..

[26]  R. P. Hettich,et al.  Semi-infinite programming: Conditions of optimality and applications , 1978 .

[27]  Jong-Shi Pang,et al.  Piecewise Sequential Quadratic Programming for Mathematical Programs with Nonlinear Complementarity Constraints , 1998 .

[28]  Marc Lassonde,et al.  Approximation, optimization and mathematical economics , 2001 .

[29]  Johan A. K. Suykens,et al.  Distributed nonlinear optimal control using sequential convex programming and smoothing techniques , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[30]  Panos M. Pardalos,et al.  Multilevel Optimization: Algorithms and Applications , 2012 .

[31]  Oliver Stein,et al.  The Adaptive Convexification Algorithm: A Feasible Point Method for Semi-Infinite Programming , 2007, SIAM J. Optim..

[32]  M. Powell Convergence properties of algorithms for nonlinear optimization , 1986 .

[33]  Sven Leyffer,et al.  Local Convergence of SQP Methods for Mathematical Programs with Equilibrium Constraints , 2006, SIAM J. Optim..

[34]  Hanif D. Sherali,et al.  New reformulation linearization/convexification relaxations for univariate and multivariate polynomial programming problems , 1997, Oper. Res. Lett..

[35]  G. Still,et al.  On Optimality Conditions for Generalized Semi-Infinite Programming Problems , 2000 .

[36]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[37]  Kenneth O. Kortanek,et al.  Semi-Infinite Programming: Theory, Methods, and Applications , 1993, SIAM Rev..

[38]  Mihai Anitescu,et al.  Global Convergence of an Elastic Mode Approach for a Class of Mathematical Programs with Complementarity Constraints , 2005, SIAM J. Optim..

[39]  Jonathan F. Bard,et al.  Practical Bilevel Optimization: Algorithms and Applications , 1998 .

[40]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[41]  Filip Logist,et al.  Approximate robust optimization of time-periodic stationary states with application to biochemical processes , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[42]  Oliver Stein,et al.  Solving Semi-Infinite Optimization Problems with Interior Point Techniques , 2003, SIAM J. Control. Optim..

[43]  M. J. D. Powell,et al.  THE CONVERGENCE OF VARIABLE METRIC METHODS FOR NONLINEARLY CONSTRAINED OPTIMIZATION CALCULATIONS , 1978 .

[44]  Robert J. Vanderbei,et al.  Interior-Point Algorithms, Penalty Methods and Equilibrium Problems , 2006, Comput. Optim. Appl..

[45]  Tim Van Voorhis,et al.  A Global Optimization Algorithm using Lagrangian Underestimates and the Interval Newton Method , 2002, J. Glob. Optim..

[46]  E. Polak,et al.  A quadratically convergent algorithm for solving infinite dimensional inequalities , 1982 .

[47]  Arkadi Nemirovski,et al.  Robust solutions of uncertain linear programs , 1999, Oper. Res. Lett..

[48]  R. Fletcher,et al.  Second order corrections for non-differentiable optimization , 1982 .

[49]  DONALD GOLDFARB,et al.  AN ACTIVE SET METHOD FOR MATHEMATICAL PROGRAMS WITH LINEAR COMPLEMENTARITY CONSTRAINTS , 2007 .

[50]  Moritz Diehl,et al.  ACADO toolkit—An open‐source framework for automatic control and dynamic optimization , 2011 .

[51]  Moritz Diehl,et al.  Loss of Superlinear Convergence for an SQP-Type Method with Conic Constraints , 2006, SIAM J. Optim..

[52]  D. Mayne,et al.  Superlinearly convergent algorithm for min-max problems , 1991 .

[53]  Hanif D. Sherali,et al.  A reformulation-convexification approach for solving nonconvex quadratic programming problems , 1995, J. Glob. Optim..

[54]  Hubertus Th. Jongen,et al.  Generalized semi-infinite optimization: A first order optimality condition and examples , 1998, Math. Program..

[55]  A. Neumaier Complete search in continuous global optimization and constraint satisfaction , 2004, Acta Numerica.

[56]  Richard D. Braatz,et al.  Open-loop and closed-loop robust optimal control of batch processes using distributional and worst-case analysis , 2004 .

[57]  Georg Still,et al.  Discretization in semi-infinite programming: the rate of convergence , 2001, Math. Program..

[58]  N. Maratos,et al.  Exact penalty function algorithms for finite dimensional and control optimization problems , 1978 .

[59]  Stephen J. Wright Modifying SQP for Degenerate Problems , 2002, SIAM J. Optim..

[60]  Stephen M. Robinson,et al.  Strongly Regular Generalized Equations , 1980, Math. Oper. Res..

[61]  Jianzhong Zhang,et al.  A New Extreme Point Algorithm and Its Application in PSQP Algorithms for Solving Mathematical Programs with Linear Complementarity Constraints , 2001, J. Glob. Optim..

[62]  Stefan Scholtes,et al.  Mathematical Programs with Complementarity Constraints: Stationarity, Optimality, and Sensitivity , 2000, Math. Oper. Res..

[63]  Gerhard-Wilhelm Weber,et al.  Generalized semi-infinite optimization and related topics , 1999 .

[64]  Z. Nagy,et al.  Distributional uncertainty analysis using power series and polynomial chaos expansions , 2007 .

[65]  Francisco Facchinei,et al.  A smoothing method for mathematical programs with equilibrium constraints , 1999, Math. Program..

[66]  Laurent El Ghaoui,et al.  Robust Solutions to Least-Squares Problems with Uncertain Data , 1997, SIAM J. Matrix Anal. Appl..

[67]  Moritz Diehl,et al.  An approximation technique for robust nonlinear optimization , 2006, Math. Program..