A survey of nonlinear robust optimization

Abstract Robust optimization (RO) has attracted much attention from the optimization community over the past decade. RO is dedicated to solving optimization problems subject to uncertainty: design constraints must be satisfied for all the values of the uncertain parameters within a given uncertainty set. Uncertainty sets may be modeled as deterministic sets (boxes, polyhedra, ellipsoids), in which case the RO problem may be reformulated via worst-case analysis, or as families of distributions. The challenge of RO is to reformulate or approximate robust constraints so that the uncertain optimization problem is transformed into a tractable deterministic optimization problem. Most reformulation methods assume linearity of the robust constraints or uncertainty sets of favorable shape, which represents only a fraction of real-world applications. This survey addresses nonlinear RO and includes problem formulations and applications, solution approaches, and available software with code samples.

[1]  Biao Huang,et al.  Nonlinear robust optimization for process design , 2018 .

[2]  Robert J. Vanderbei,et al.  Robust Optimization of Large-Scale Systems , 1995, Oper. Res..

[3]  L. Huyse,et al.  Robust airfoil optimization to achieve drag reduction over a range of Mach numbers , 2002 .

[4]  Lorenz T. Biegler,et al.  An Interior Point Method for Mathematical Programs with Complementarity Constraints (MPCCs) , 2005, SIAM J. Optim..

[5]  Ruiwei Jiang,et al.  Data-driven chance constrained stochastic program , 2015, Mathematical Programming.

[6]  Jean-Philippe Vial,et al.  Deriving robust counterparts of nonlinear uncertain inequalities , 2012, Math. Program..

[7]  Sven Leyffer,et al.  Local Convergence of SQP Methods for Mathematical Programs with Equilibrium Constraints , 2006, SIAM J. Optim..

[8]  Andrew R. Conn,et al.  Bilevel derivative-free optimization and its application to robust optimization , 2012, Optim. Methods Softw..

[9]  Laurent El Ghaoui,et al.  Robust Solutions to Uncertain Semidefinite Programs , 1998, SIAM J. Optim..

[10]  Daniel Kuhn,et al.  Worst-Case Value at Risk of Nonlinear Portfolios , 2013, Manag. Sci..

[11]  Laurent El Ghaoui,et al.  Worst-Case Value-At-Risk and Robust Portfolio Optimization: A Conic Programming Approach , 2003, Oper. Res..

[12]  Dimitris Bertsimas,et al.  Robust Optimization for Unconstrained Simulation-Based Problems , 2010, Oper. Res..

[13]  Peter W. Glynn,et al.  Likelihood robust optimization for data-driven problems , 2013, Computational Management Science.

[14]  P. Parrilo,et al.  Symmetry groups, semidefinite programs, and sums of squares , 2002, math/0211450.

[15]  Moritz Diehl,et al.  An approximation technique for robust nonlinear optimization , 2006, Math. Program..

[16]  Jorge Nocedal,et al.  Interior Methods for Mathematical Programs with Complementarity Constraints , 2006, SIAM J. Optim..

[17]  Daniel Kuhn,et al.  Distributionally robust joint chance constraints with second-order moment information , 2011, Mathematical Programming.

[18]  S. Leyffer Complementarity constraints as nonlinear equations: Theory and numerical experience , 2006 .

[19]  Constantine Caramanis,et al.  Theory and Applications of Robust Optimization , 2010, SIAM Rev..

[20]  Warren Hare,et al.  A derivative-free approximate gradient sampling algorithm for finite minimax problems , 2013, Comput. Optim. Appl..

[21]  Huan Xu,et al.  Distributionally robust chance constraints for non-linear uncertainties , 2014, Mathematical Programming.

[22]  Bernhard Sendhoff,et al.  Robust Optimization - A Comprehensive Survey , 2007 .

[23]  J. J. Bisschop,et al.  AIMMS : The Modeling System , 1993 .

[24]  Moritz Diehl,et al.  Nonlinear MPC of kites under varying wind conditions for a new class of large‐scale wind power generators , 2007 .

[25]  J. Lofberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004, 2004 IEEE International Conference on Robotics and Automation (IEEE Cat. No.04CH37508).

[26]  D. Bertsimas,et al.  Moment Problems and Semidefinite Optimization , 2000 .

[27]  Jan H. Maruhn,et al.  A successive SDP-NSDP approach to a robust optimization problem in finance , 2009, Comput. Optim. Appl..

[28]  D. D. Hertog,et al.  Robust nonlinear optimization via the dual , 2015 .

[29]  Dimitris Bertsimas,et al.  Robust optimization with simulated annealing , 2010, J. Glob. Optim..

[30]  Arkadi Nemirovski,et al.  Robust solutions of Linear Programming problems contaminated with uncertain data , 2000, Math. Program..

[31]  Daniel Kuhn,et al.  Robust portfolio optimization with derivative insurance guarantees , 2011, Eur. J. Oper. Res..

[32]  Amir Beck,et al.  Duality in robust optimization: Primal worst equals dual best , 2009, Oper. Res. Lett..

[33]  Martin Mönnigmann,et al.  Numerical solution approaches for robust nonlinear optimal control problems , 2008, Comput. Chem. Eng..

[34]  Garud Iyengar,et al.  A new method of channel shortening with applications to discrete multi-tone (DMT) systems , 1998, ICC '98. 1998 IEEE International Conference on Communications. Conference Record. Affiliated with SUPERCOMM'98 (Cat. No.98CH36220).

[35]  Constantine Caramanis,et al.  Robust PCA via Outlier Pursuit , 2010, IEEE Transactions on Information Theory.

[36]  Oliver Stein,et al.  The adaptive convexification algorithm for semi-infinite programming with arbitrary index sets , 2012, Math. Program..

[37]  J. Löfberg Modeling and solving uncertain optimization problems in YALMIP , 2008 .

[38]  Daniel Kuhn,et al.  Distributionally Robust Convex Optimization , 2014, Oper. Res..

[39]  R. Steele Optimization , 2005 .

[40]  Oliver Lass,et al.  An approach for robust PDE-constrained optimization with application to shape optimization of electrical engines and of dynamic elastic structures under uncertainty , 2018, Optimization and Engineering.

[41]  Cécile Murat,et al.  Recent advances in robust optimization: An overview , 2014, Eur. J. Oper. Res..

[42]  Stefan M. Wild,et al.  Derivative-free robust optimization by outer approximations , 2018, Math. Program..

[43]  ScheelHolger,et al.  Mathematical Programs with Complementarity Constraints , 2000 .

[44]  Yinyu Ye,et al.  Distributionally Robust Optimization Under Moment Uncertainty with Application to Data-Driven Problems , 2010, Oper. Res..

[45]  Stephen P. Boyd,et al.  A minimax theorem with applications to machine learning, signal processing, and finance , 2007, 2007 46th IEEE Conference on Decision and Control.

[46]  Jean-Philippe Vial,et al.  Robust Optimization , 2021, ICORES.

[47]  Iain Dunning,et al.  Reformulation versus cutting-planes for robust optimization , 2015, Comput. Manag. Sci..

[48]  Andreas Griewank,et al.  Algorithmic differentiation for piecewise smooth functions: a case study for robust optimization , 2018, Optim. Methods Softw..

[49]  Johan Löfberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004 .

[50]  Donald Goldfarb,et al.  Robust convex quadratically constrained programs , 2003, Math. Program..

[51]  Huseyin Topaloglu,et al.  Robust Assortment Optimization in Revenue Management Under the Multinomial Logit Choice Model , 2012, Oper. Res..

[52]  Harvey J. Greenberg,et al.  The Operations Research Revolution , 2015 .

[53]  Omid Nohadani,et al.  Optimization under Decision-Dependent Uncertainty , 2016, SIAM J. Optim..

[54]  R. Reemtsen Some outer approximation methods for semi-infinite optimization problems , 1994 .

[55]  Marc Goerigk,et al.  ROPI—a robust optimization programming interface for C++ , 2014, Optim. Methods Softw..

[56]  Dick den Hertog,et al.  Hidden conic quadratic representation of some nonconvex quadratic optimization problems , 2014, Math. Program..

[57]  Matthijs Langelaar,et al.  Expected improvement based infill sampling for global robust optimization of constrained problems , 2017 .

[58]  Oliver Lass,et al.  Model Order Reduction Techniques with a Posteriori Error Control for Nonlinear Robust Optimization Governed by Partial Differential Equations , 2017, SIAM J. Sci. Comput..

[59]  Marianthi G. Ierapetritou,et al.  Process scheduling under uncertainty: Review and challenges , 2008, Comput. Chem. Eng..

[60]  Michael P. Friedlander,et al.  A two-sided relaxation scheme for Mathematical Programs with Equilibrium Constraints , 2005, SIAM J. Optim..

[61]  Arkadi Nemirovski,et al.  Robust Truss Topology Design via Semidefinite Programming , 1997, SIAM J. Optim..

[62]  Arkadi Nemirovski,et al.  Robust Convex Optimization , 1998, Math. Oper. Res..

[63]  G. Calafiore,et al.  On Distributionally Robust Chance-Constrained Linear Programs , 2006 .

[64]  Melvyn Sim,et al.  Robust Optimization Made Easy with ROME , 2011, Oper. Res..

[65]  Sven Leyffer,et al.  Solving mathematical programs with complementarity constraints as nonlinear programs , 2004, Optim. Methods Softw..

[66]  Aurélie Thiele,et al.  Short sales in Log-robust portfolio management , 2011, Eur. J. Oper. Res..

[67]  Chiranjib Bhattacharyya,et al.  Chance constrained uncertain classification via robust optimization , 2011, Math. Program..

[68]  Alexander Shapiro,et al.  Convex Approximations of Chance Constrained Programs , 2006, SIAM J. Optim..

[69]  Laurent El Ghaoui,et al.  Robust Control of Markov Decision Processes with Uncertain Transition Matrices , 2005, Oper. Res..

[70]  Arkadi Nemirovski,et al.  Robust solutions of uncertain linear programs , 1999, Oper. Res. Lett..

[71]  Theodore B. Trafalis,et al.  Robust support vector machines for classification and computational issues , 2007, Optim. Methods Softw..

[72]  A. Kleywegt,et al.  Distributionally Robust Stochastic Optimization with Wasserstein Distance , 2016, Math. Oper. Res..

[73]  Shie Mannor,et al.  Robust Regression and Lasso , 2008, IEEE Transactions on Information Theory.

[74]  Matteo Fischetti,et al.  Cutting plane versus compact formulations for uncertain (integer) linear programs , 2012, Math. Program. Comput..

[75]  Melvyn Sim,et al.  Distributionally Robust Optimization and Its Tractable Approximations , 2010, Oper. Res..

[76]  Edite M. G. P. Fernandes,et al.  SIPAMPL: Semi-infinite programming with AMPL , 2004, TOMS.

[77]  Alexander Shapiro,et al.  Distributionally Robust Stochastic Programming , 2017, SIAM J. Optim..

[78]  Warren Hare,et al.  Derivative-free optimization methods for finite minimax problems , 2013, Optim. Methods Softw..

[79]  Bertrand Melenberg,et al.  Robust Optimization with Ambiguous Stochastic Constraints Under Mean and Dispersion Information , 2018, Oper. Res..

[80]  Dick den Hertog,et al.  A survey of adjustable robust optimization , 2019, Eur. J. Oper. Res..

[81]  Bertrand Melenberg,et al.  Computationally Tractable Counterparts of Distributionally Robust Constraints on Risk Measures , 2014, SIAM Rev..

[82]  Anja De Waegenaere,et al.  Robust Solutions of Optimization Problems Affected by Uncertain Probabilities , 2011, Manag. Sci..

[83]  Robert J. Vanderbei,et al.  Interior-Point Algorithms, Penalty Methods and Equilibrium Problems , 2006, Comput. Optim. Appl..

[84]  K. M. Teo,et al.  Robust optimization in electromagnetic scattering problems , 2007 .

[85]  Arkadi Nemirovski,et al.  Robust optimization – methodology and applications , 2002, Math. Program..

[86]  E. Polak,et al.  An Outer Approximations Approach to Reliability-Based Optimal Design of Structures , 1998 .

[87]  Alexander Mitsos,et al.  Robust Dynamic Optimization of a Semi-Batch Emulsion Polymerization Process with Parametric Uncertainties - A Heuristic Approach , 2016 .

[88]  Shie Mannor,et al.  A distributional interpretation of robust optimization , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[89]  F. Clarke Optimization And Nonsmooth Analysis , 1983 .

[90]  Fabio Celani,et al.  A Robust Optimization Approach for Magnetic Spacecraft Attitude Stabilization , 2016, Journal of Optimization Theory and Applications.

[91]  Shie Mannor,et al.  Robustness and Regularization of Support Vector Machines , 2008, J. Mach. Learn. Res..

[92]  Herbert E. Scarf,et al.  A Min-Max Solution of an Inventory Problem , 1957 .

[93]  Vishal Gupta,et al.  Data-driven robust optimization , 2013, Math. Program..

[94]  Shabnam Rasoulian,et al.  Uncertainty analysis and robust optimization of multiscale process systems with application to epitaxial thin film growth , 2014 .

[95]  Georgia Perakis,et al.  A Robust SQP Method for Mathematical Programs with Linear Complementarity Constraints , 2006, Comput. Optim. Appl..

[96]  Carsten W. Scherer,et al.  Matrix Sum-of-Squares Relaxations for Robust Semi-Definite Programs , 2006, Math. Program..

[97]  Donald Goldfarb,et al.  Robust Portfolio Selection Problems , 2003, Math. Oper. Res..

[98]  Oliver Stein,et al.  Bi-Level Strategies in Semi-Infinite Programming , 2003 .

[99]  Daniel Kuhn,et al.  Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations , 2015, Mathematical Programming.

[100]  Oliver Stein,et al.  Solving Semi-Infinite Optimization Problems with Interior Point Techniques , 2003, SIAM J. Control. Optim..

[101]  Francisco J. Nogales,et al.  Portfolio Selection With Robust Estimation , 2007, Oper. Res..

[102]  Dick den Hertog,et al.  Safe Approximations of Ambiguous Chance Constraints Using Historical Data , 2013, INFORMS J. Comput..

[103]  Stefano Lucidi,et al.  Derivative-Free Robust Optimization for Circuit Design , 2015, J. Optim. Theory Appl..

[104]  S. Nash,et al.  Linear and Nonlinear Optimization , 2008 .

[105]  Mihai Anitescu,et al.  Global Convergence of an Elastic Mode Approach for a Class of Mathematical Programs with Complementarity Constraints , 2005, SIAM J. Optim..

[106]  Bertrand Melenberg,et al.  Extending the Scope of Robust Quadratic Optimization , 2017, INFORMS J. Comput..

[107]  Dick den Hertog,et al.  A practical guide to robust optimization , 2015, 1501.02634.

[108]  Kenneth O. Kortanek,et al.  Semi-Infinite Programming: Theory, Methods, and Applications , 1993, SIAM Rev..

[109]  J. Outrata On mathematical programs with complementarity constraints , 2000 .

[110]  Aurélie Thiele,et al.  A log-robust optimization approach to portfolio management , 2011, OR Spectr..

[111]  R. Rockafellar,et al.  Optimization of conditional value-at risk , 2000 .

[112]  Melvyn Sim,et al.  Adaptive Distributionally Robust Optimization , 2019, Manag. Sci..

[113]  Melvyn Sim,et al.  TRACTABLE ROBUST EXPECTED UTILITY AND RISK MODELS FOR PORTFOLIO OPTIMIZATION , 2009 .

[114]  Kai Ye,et al.  Robust portfolio optimization: a conic programming approach , 2012, Comput. Optim. Appl..

[115]  Erick Delage,et al.  Robust Multistage Decision Making , 2015 .

[116]  Stefan Ulbrich,et al.  A Second Order Approximation Technique for Robust Shape Optimization , 2011 .

[117]  Elaine Hale,et al.  Case Studies For a First-Order Fobust Nonlinear Programming , 2006 .

[118]  Yin Zhang,et al.  Case Studies for a First-Order Robust Nonlinear Programming Formulation , 2007 .