Aspects of quadratic optimization - nonconvexity, uncertainty, and applications

Quadratic Optimization (QO) has been studied extensively in the literature due to its application in real-life problems. This thesis deals with two complicated aspects of QO problems, namely nonconvexity and uncertainty. A nonconvex QO problem is intractable in general. The first part of this thesis presents methods to approximate a nonconvex QP problem. Another important aspect of a QO problem is taking into account uncertainties in the parameters since they are mostly approximated/estimated from data. The second part of the thesis contains analyses of two methods that deal with uncertainties in a convex QO problem, namely Static and Adjustable Robust Optimization problems. To test the methods proposed in this thesis, the following three real-life applications have been considered: pooling problem, portfolio problem, and norm approximation problem.

[1]  Aharon Ben-Tal,et al.  Lectures on modern convex optimization , 1987 .

[2]  Dimitris Bertsimas,et al.  On the performance of affine policies for two-stage adaptive optimization: a geometric perspective , 2015, Math. Program..

[3]  Gerardo Toraldo,et al.  On the Solution of Large Quadratic Programming Problems with Bound Constraints , 1991, SIAM J. Optim..

[4]  T. E. Baker,et al.  Successive Linear Programming at Exxon , 1985 .

[5]  Bin Ran,et al.  Discrete Optimal Control and Nonlinear Programming , 1996 .

[6]  Duan Li,et al.  Nonconvex quadratically constrained quadratic programming: best D.C. decompositions and their SDP representations , 2011, J. Glob. Optim..

[7]  Aharon Ben-Tal,et al.  Global minimization by reducing the duality gap , 1994, Math. Program..

[8]  Marc Teboulle,et al.  Hidden convexity in some nonconvex quadratically constrained quadratic programming , 1996, Math. Program..

[9]  L. Foulds,et al.  A bilinear approach to the pooling problem , 1992 .

[10]  Björn E. Ottersten,et al.  An iterative approach to nonconvex QCQP with applications in signal processing , 2016, 2016 IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM).

[11]  清水 邦夫.  Continuous Univariate Distributions Volume 1/N.L.Johnson,S.Kotz,N.Balakrishnan(1994) , 1995 .

[12]  Jean B. Lasserre,et al.  Polynomial Programming: LP-Relaxations Also Converge , 2005, SIAM J. Optim..

[13]  Yinyu Ye,et al.  Distributionally Robust Optimization Under Moment Uncertainty with Application to Data-Driven Problems , 2010, Oper. Res..

[14]  Ali Elkamel,et al.  Planning and Integration of Refinery and Petrochemical Operations: AL-QAHTANI:REFINERY O-BK , 2010 .

[15]  F. Fabozzi.  Robust Portfolio Optimization and Management , 2007 .

[16]  Katta G. Murty,et al.  Linear complementarity, linear and nonlinear programming , 1988 .

[17]  Mohammed Alfaki,et al.  Solving the pooling problem with LMI relaxations , 2012 .

[18]  Anja De Waegenaere,et al.  Robust Solutions of Optimization Problems Affected by Uncertain Probabilities , 2011, Manag. Sci..

[19]  A. Berman,et al.  Completely Positive Matrices , 2003 .

[20]  Duan Li,et al.  Convex Relaxations with Second Order Cone Constraints for Nonconvex Quadratically Constrained Quadratic Programming , 2016 .

[21]  Alan Edelman,et al.  Julia: A Fresh Approach to Numerical Computing , 2014, SIAM Rev..

[22]  Jean B. Lasserre,et al.  Sparse-BSOS: a bounded degree SOS hierarchy for large scale polynomial optimization with sparsity , 2016, Mathematical Programming Computation.

[23]  B. Peyton,et al.  An Introduction to Chordal Graphs and Clique Trees , 1993 .

[24]  Nikolaos V. Sahinidis,et al.  Convexification and Global Optimization in Continuous and Mixed-Integer Nonlinear Programming , 2002 .

[25]  Dimitris Bertsimas,et al.  On the power and limitations of affine policies in two-stage adaptive optimization , 2012, Math. Program..

[26]  P. Parrilo,et al.  From coefficients to samples: a new approach to SOS optimization , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[27]  Jean B. Lasserre,et al.  Global Optimization with Polynomials and the Problem of Moments , 2000, SIAM J. Optim..

[28]  Stephen P. Boyd,et al.  Applications of second-order cone programming , 1998 .

[29]  T. E. Baker,et al.  A History of Mathematical Programming in the Petroleum Industry , 1990 .

[30]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[31]  Panos M. Pardalos,et al.  Quadratic programming with one negative eigenvalue is NP-hard , 1991, J. Glob. Optim..

[32]  Johannes Bisschop,et al.  AIMMS - Optimization Modeling , 2006 .

[33]  N. Sahinidis,et al.  A Lagrangian Approach to the Pooling Problem , 1999 .

[34]  Isak Nielsen,et al.  Structure-Exploiting Numerical Algorithms for Optimal Control , 2017 .

[35]  Ignacio E. Grossmann,et al.  Global optimization for the synthesis of integrated water systems in chemical processes , 2006, Comput. Chem. Eng..

[36]  Ruth Misener,et al.  Piecewise parametric structure in the pooling problem: from sparse strongly-polynomial solutions to NP-hardness , 2017, Journal of Global Optimization.

[37]  Patrick R. Amestoy,et al.  An Approximate Minimum Degree Ordering Algorithm , 1996, SIAM J. Matrix Anal. Appl..

[38]  Tamás Terlaky,et al.  A Survey of the S-Lemma , 2007, SIAM Rev..

[39]  Antonio De Maio,et al.  Semidefinite programming, matrix decomposition, and radar code design , 2010, Convex Optimization in Signal Processing and Communications.

[40]  Hussein Naseraldin,et al.  Facility Location: A Robust Optimization Approach , 2011 .

[41]  Kees Roos,et al.  Robust Solutions of Uncertain Quadratic and Conic-Quadratic Problems , 2002, SIAM J. Optim..

[42]  Masakazu Kojima,et al.  Generalized Lagrangian Duals and Sums of Squares Relaxations of Sparse Polynomial Optimization Problems , 2005, SIAM J. Optim..

[43]  Mohammed Alfaki,et al.  Models and Solution Methods for the Pooling Problem , 2012 .

[44]  Yonina C. Eldar,et al.  Strong Duality in Nonconvex Quadratic Optimization with Two Quadratic Constraints , 2006, SIAM J. Optim..

[45]  Arnold Neumaier,et al.  Introduction to Numerical Analysis , 2001 .

[46]  Kurt M. Anstreicher,et al.  On convex relaxations for quadratically constrained quadratic programming , 2012, Math. Program..

[47]  Etienne de Klerk,et al.  The complexity of optimizing over a simplex, hypercube or sphere: a short survey , 2008, Central Eur. J. Oper. Res..

[48]  V. Barnett,et al.  Applied Linear Statistical Models , 1975 .

[49]  Terry L. Friesz.  Nonlinear Programming and Discrete-Time Optimal Control , 2010 .

[50]  C. A. Haverly.  Studies of the behavior of recursion for the pooling problem , 1978, SMAP.

[51]  Mayank Sharma,et al.  Supermodularity and Affine Policies in Dynamic Robust Optimization , 2013, Oper. Res..

[52]  Jiming Peng,et al.  Primal-Dual Interior-Point Methods for Second-Order Conic Optimization Based on Self-Regular Proximities , 2002, SIAM J. Optim..

[53]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[54]  Mohammed Alfaki,et al.  A cost minimization heuristic for the pooling problem , 2014, Ann. Oper. Res..

[55]  Amir Ali Ahmadi,et al.  Some applications of polynomial optimization in operations research and real-time decision making , 2015, Optimization Letters.

[56]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms , 1993 .

[57]  Masakazu Muramatsu,et al.  Sums of Squares and Semidefinite Programming Relaxations for Polynomial Optimization Problems with Structured Sparsity , 2004 .

[58]  Dimitris Bertsimas,et al.  On the approximability of adjustable robust convex optimization under uncertainty , 2013, Math. Methods Oper. Res..

[59]  Christodoulos A. Floudas,et al.  ADVANCES FOR THE POOLING PROBLEM: MODELING, GLOBAL OPTIMIZATION, AND COMPUTATIONAL STUDIES , 2009 .

[60]  Laurent El Ghaoui,et al.  Robust Solutions to Least-Squares Problems with Uncertain Data , 1997, SIAM J. Matrix Anal. Appl..

[61]  Dag Haugland,et al.  The computational complexity of the pooling problem , 2016, J. Glob. Optim..

[62]  Vishal Gupta,et al.  Data-driven robust optimization , 2013, Math. Program..

[63]  A. Ben-Tal,et al.  Adjustable robust solutions of uncertain linear programs , 2004, Math. Program..

[64]  Shuzhong Zhang,et al.  On Cones of Nonnegative Quadratic Functions , 2003, Math. Oper. Res..

[65]  Arkadi Nemirovski,et al.  Robust solutions of uncertain linear programs , 1999, Oper. Res. Lett..

[66]  R. A. Nicolaides,et al.  On a Class of Finite Elements Generated by Lagrange Interpolation , 1972 .

[67]  Nalan Gülpinar,et al.  Worst-case robust decisions for multi-period mean-variance portfolio optimization , 2007, Eur. J. Oper. Res..

[68]  Vahab S. Mirrokni,et al.  Robust Combinatorial Optimization with Exponential Scenarios , 2007, IPCO.

[69]  Dimitris Bertsimas,et al.  A tight characterization of the performance of static solutions in two-stage adjustable robust linear optimization , 2014, Math. Program..

[70]  D. D. Hertog,et al.  Robust nonlinear optimization via the dual , 2015 .

[71]  Jean B. Lasserre,et al.  A bounded degree SOS hierarchy for polynomial optimization , 2015, EURO J. Comput. Optim..

[72]  Daniel Bienstock,et al.  Polynomial Solvability of Variants of the Trust-Region Subproblem , 2014, SODA.

[73]  FAIZ A. AL-KHAYYAL,et al.  A relaxation method for nonconvex quadratically constrained quadratic programs , 1995, J. Glob. Optim..

[74]  Aharon Ben-Tal,et al.  Adjustable robust counterpart of conic quadratic problems , 2008, Math. Methods Oper. Res..

[75]  Richard W. Cottle,et al.  Linear Complementarity Problem. , 1992 .

[76]  Jorge J. Moré,et al.  The NEOS Server , 1998 .

[77]  Nikos D. Sidiropoulos,et al.  Hidden Convexity in QCQP with Toeplitz-Hermitian Quadratics , 2015, IEEE Signal Processing Letters.

[78]  Stephen P. Boyd,et al.  General Heuristics for Nonconvex Quadratically Constrained Quadratic Programming , 2017, 1703.07870.

[79]  Etienne de Klerk,et al.  A numerical evaluation of the bounded degree sum-of-squares hierarchy of Lasserre, Toh, and Yang on the pooling problem , 2018, Ann. Oper. Res..

[80]  R. Tütüncü,et al.  Adjustable Robust Optimization Models for a Nonlinear Two-Period System , 2008 .

[81]  Amir Ardestani-Jaafari,et al.  The Value of Flexibility in Robust Location-Transportation Problems , 2014, Transp. Sci..

[82]  J. Krivine,et al.  Anneaux préordonnés , 1964 .

[83]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[84]  Lorenzo Trapani.  Testing for (in)finite moments , 2016 .

[85]  Dick den Hertog,et al.  A practical guide to robust optimization , 2015, 1501.02634.

[86]  Nikolaos Trichakis,et al.  Pareto Efficiency in Robust Optimization , 2014, Manag. Sci..

[87]  L. Grippo,et al.  A Class of Structured Quasi-Newton Algorithms for Optimal Control Problems , 1983 .

[88]  Myun-Seok Cheon,et al.  Relaxations and discretizations for the pooling problem , 2016, Journal of Global Optimization.

[89]  Donald Goldfarb,et al.  Robust convex quadratically constrained programs , 2003, Math. Program..

[90]  Kazuo Murota,et al.  Exploiting Sparsity in Semidefinite Programming via Matrix Completion I: General Framework , 2000, SIAM J. Optim..

[91]  Augusto Aubry,et al.  Radar waveform design in a spectrally crowded environment via nonconvex quadratic optimization , 2014, IEEE Transactions on Aerospace and Electronic Systems.

[92]  Stephen P. Boyd,et al.  Variations and extension of the convex–concave procedure , 2016 .

[93]  Boaz Golany,et al.  On the average performance of the adjustable RO and its use as an offline tool for multi-period production planning under uncertainty , 2016, Comput. Manag. Sci..

[94]  Mohammed Alfaki,et al.  Strong formulations for the pooling problem , 2013, J. Glob. Optim..

[95]  Jean-Philippe Vial,et al.  Deriving robust counterparts of nonlinear uncertain inequalities , 2012, Math. Program..

[96]  Santanu S. Dey,et al.  Analysis of MILP Techniques for the Pooling Problem , 2015, Oper. Res..

[97]  Stephen P. Boyd,et al.  A rank minimization heuristic with application to minimum order system approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[98]  M. Laurent.  Sums of Squares, Moment Matrices and Optimization Over Polynomials , 2009 .

[99]  Jason Rife,et al.  The Effect of Uncertain Covariance on a Chi-Square Integrity Monitor , 2013 .

[100]  D. O’Leary.  A generalized conjugate gradient algorithm for solving a class of quadratic programming problems , 1977 .

[101]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[102]  Donald Goldfarb,et al.  Robust Portfolio Selection Problems , 2003, Math. Oper. Res..

[103]  Vaithilingam Jeyakumar,et al.  Robust SOS-convex polynomial optimization problems: exact SDP relaxations , 2013, Optimization Letters.

[104]  Zhi-You Wu,et al.  Peeling Off a Nonconvex Cover of an Actual Convex Problem: Hidden Convexity , 2007, SIAM J. Optim..

[105]  Laurent El Ghaoui,et al.  Robust Optimization , 2021, ICORES.

[106]  Jean B. Lasserre.  A Lagrangian Relaxation View of Linear and Semidefinite Hierarchies , 2013, SIAM J. Optim..

[107]  Mei Han An,et al.  accuracy and stability of numerical algorithms , 1991 .

[108]  Shuzhong Zhang,et al.  New Results on Quadratic Minimization , 2003, SIAM J. Optim..

[109]  Thomas F. Coleman,et al.  An efficient trust region method for unconstrained discrete-time optimal control problems , 1995, Comput. Optim. Appl..

[110]  Arkadi Nemirovski,et al.  Robust Convex Optimization , 1998, Math. Oper. Res..

[111]  Martin S. Andersen,et al.  Chordal Graphs and Semidefinite Optimization , 2015, Found. Trends Optim..

[112]  Pablo A. Parrilo,et al.  Optimality of Affine Policies in Multistage Robust Optimization , 2009, Math. Oper. Res..

[113]  Christodoulos A. Floudas,et al.  APOGEE: Global optimization of standard, generalized, and extended pooling problems via linear and logarithmic partitioning schemes , 2011, Comput. Chem. Eng..

[114]  Erling D. Andersen,et al.  On implementing a primal-dual interior-point method for conic quadratic optimization , 2003, Math. Program..

[115]  N. Cristianini,et al.  Estimating the moments of a random vector with applications , 2003 .