Reformulation and convex relaxation techniques for global optimization

Abstract.We survey the main results obtained by the author in his PhD dissertation supervised by Prof. Costas Pantelides. It was defended at the Imperial College, London. The thesis is written in English and is available from http://or.elet.polimi.it/people/Liberti/phdtesis.ps.gz. The most widely employed deterministic method for the global solution of nonconvex NLPs and MINLPs is the spatial Branch-and-Bound (sBB) algorithm, and one of its most crucial steps is the computation of the lower bound at each sBB node. We investigate different reformulations of the problem so that the resulting convex relaxation is tight. In particular, we suggest a novel technique for reformulating a wide class of bilinear problems so that some of the bilinear terms are replaced by linear constraints. Moreover, an in-depth analysis of a convex envelope for piecewise-convex and concave terms is performed. All the proposed algorithms were implemented in $oo{\cal OPS}$, an object-oriented callable library for constructing MINLPs in structured form and solving them using a variety of local and global solvers.

[1]  Iiro Harjunkoski,et al.  An extended cutting plane method for a class of non-convex MINLP problems , 1998 .

[2]  Nils Tönshoff,et al.  Implementation and Computational Results , 1997 .

[3]  C. C. Meewella,et al.  An algorithm for global optimization of Lipschitz continuous functions , 1988 .

[4]  Nikolaos V. Sahinidis,et al.  Global Optimization of Multiplicative Programs , 2003, J. Glob. Optim..

[5]  Christodoulos A. Floudas,et al.  New Formulations and Branching Strategies for the GOP Algorithm , 1996 .

[6]  Ross E. Swaney,et al.  Global optimization of nonconvex nonlinear programs using parallel branch and bound , 1995 .

[7]  Frank Harary,et al.  Graph Theory , 2016 .

[8]  Pierre Hansen,et al.  An analytical approach to global optimization , 1991, Math. Program..

[9]  Harold P. Benson,et al.  Concave Minimization: Theory, Applications and Algorithms , 1995 .

[10]  Hanif D. Sherali,et al.  Tight Relaxations for Nonconvex Optimization Problems Using the Reformulation-Linearization/Convexification Technique (RLT) , 2002 .

[11]  Lonnie Hamm,et al.  GLOBAL OPTIMIZATION METHODS , 2002 .

[12]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[13]  Roman G. Strongin,et al.  Algorithms for multi-extremal mathematical programming problems employing the set of joint space-filling curves , 1992, J. Glob. Optim..

[14]  Patrick Siarry,et al.  Tabu Search applied to global optimization , 2000, Eur. J. Oper. Res..

[15]  Ignacio E. Grossmann,et al.  An automated modelling and reformulation system for design optimization , 1992 .

[16]  Aimo A. Törn,et al.  Global Optimization , 1999, Science.

[17]  I. Grossmann,et al.  Global optimization of nonconvex mixed-integer nonlinear programming (MINLP) problems in process synthesis , 1988 .

[18]  Tapio Westerlund,et al.  Automated reformulation of disjunctive constraints in MINLP optimization , 1999 .

[19]  Hoang Tuy,et al.  D.C. Optimization: Theory, Methods and Algorithms , 1995 .

[20]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[21]  Panos M. Pardalos,et al.  The maximum clique problem , 1994, J. Glob. Optim..

[22]  G. Nemhauser,et al.  Integer Programming , 2020 .

[23]  Saumyendra Sengupta,et al.  GLOBAL CONSTRAINED OPTIMIZATION USING INTERVAL ANALYSIS , 1980 .

[24]  Edward M. B. Smith,et al.  A symbolic reformulation/spatial branch-and-bound algorithm for the global optimisation of nonconvex MINLPs , 1999 .

[25]  Nikolaos V. Sahinidis,et al.  Global optimization of mixed-integer nonlinear programs: A theoretical and computational study , 2004, Math. Program..

[26]  P. Pardalos,et al.  Handbook of global optimization , 1995 .

[27]  J. Lagrange.  Théorie des fonctions analytiques , 1967 .

[28]  F. Schoen.  TWO-PHASE METHODS FOR GLOBAL OPTIMIZATION , 2002 .

[29]  G. Isac.  Complementarity Problems , 1992 .

[30]  Tsu-Shuan Chang,et al.  A Multivariate Global Optimization Using Linear Bounding Functions , 1998, J. Glob. Optim..

[31]  N. Sahinidis,et al.  A Lagrangian Approach to the Pooling Problem , 1999 .

[32]  Renpu Ge.  A parallel global optimization algorithm for rational separable-factorable functions , 1989 .

[33]  Hanif D. Sherali,et al.  Reduced first-level representations via the reformulation-linearization technique: results, counterexamples, and computations , 2000, Discret. Appl. Math..

[34]  Leo Liberti.  Performance Comparison of Function Evaluation Methods , 2002, ArXiv.

[35]  R. Horst.  Deterministic global optimization with partition sets whose feasibility is not known: Application to concave minimization, reverse convex constraints, DC-programming, and Lipschitzian optimization , 1988 .

[36]  Eric S. Fraga,et al.  Interval Analysis in Automated Design for Bounded Solutions , 2001 .

[37]  J. Pintér.  Global optimization on convex sets , 1986 .

[38]  C. Pantelides.  The consistent intialization of differential-algebraic systems , 1988 .

[39]  A. Fischer.  New Constrained Optimization Reformulation of Complementarity Problems , 1998 .

[40]  Bruce W. Lamar,et al.  A Method for Converting a Class of Univariate Functions into d.c. Functions , 1999, J. Glob. Optim..

[41]  J. E. Falk,et al.  An Algorithm for Separable Nonconvex Programming Problems , 1969 .

[42]  T. H. I. Jaakola,et al.  Optimization by direct search and systematic reduction of the size of search region , 1973 .

[43]  Tao Wang,et al.  Efficient and Adaptive Lagrange-Multiplier Methods for Nonlinear Continuous Global Optimization , 1999, J. Glob. Optim..

[44]  George B. Dantzig,et al.  Linear programming and extensions , 1965 .

[45]  Sanguthevar Rajasekaran,et al.  On Simulated Annealing and Nested Annealing , 2000, J. Glob. Optim..

[46]  A. Neumaier,et al.  A global optimization method, αBB, for general twice-differentiable constrained NLPs — I. Theoretical advances , 1998 .

[47]  Raimund K. Ege.  Object-Oriented Programming Languages , 1992, TOOLS.

[48]  T. Westerlund,et al.  Convexification of different classes of non-convex MINLP problems , 1999 .

[49]  R. Horst,et al.  On finding new vertices and redundant constraints in cutting plane algorithms for global optimization , 1988 .

[50]  Elizabeth Eskow,et al.  Mathematical modeling of a parallel global optimization algorithm , 1989, Parallel Comput..

[51]  Duan Li,et al.  Local Convexification of the Lagrangian Function in Nonconvex Optimization , 2000 .

[52]  Christodoulos A. Floudas,et al.  A global optimization method, αBB, for process design , 1996 .

[53]  K. G. Murty,et al.  Complementarity problems , 2000 .

[54]  Christodoulos A. Floudas,et al.  Finding all solutions of nonlinearly constrained systems of equations , 1995, J. Glob. Optim..

[55]  Julius Zilinskas,et al.  Evaluation Ranges of Functions using Balanced Random Interval Arithmetic , 2003, Informatica.

[56]  R. Soland.  An Algorithm for Separable Nonconvex Programming Problems II: Nonconvex Constraints , 1971 .

[57]  B. Goertzel,et al.  Global optimization with space-filling curves , 1999 .

[58]  Edgar G. Goodaire,et al.  Discrete Mathematics With Graph Theory , 1997 .

[59]  Christodoulos A. Floudas,et al.  Deterministic Global Optimization , 1990 .

[60]  Iain S. Duff,et al.  On Algorithms for Obtaining a Maximum Transversal , 1981, TOMS.

[61]  M. Kline.  Mathematical Thought from Ancient to Modern Times , 1972 .

[62]  R. Horst.  On consistency of bounding operations in deterministic global optimization , 1989 .

[63]  Nikolaos V. Sahinidis,et al.  Exact Algorithms for Global Optimization of Mixed-Integer Nonlinear Programs , 2002 .

[64]  Reiner Horst,et al.  Convergence and restart in branch-and-bound algorithms for global optimization. Application to concave minimization and D.C. Optimization problems , 1988, Math. Program..

[65]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[66]  Nikolaos V. Sahinidis,et al.  A branch-and-reduce approach to global optimization , 1996, J. Glob. Optim..

[67]  G. Bekey,et al.  A global optimization algorithm using adaptive random search , 1980 .

[68]  P. Pardalos,et al.  Checking local optimality in constrained quadratic programming is NP-hard , 1988 .

[69]  H. Tuy.  Convex analysis and global optimization , 1998 .

[70]  Efstratios N. Pistikopoulos,et al.  A Reduced Space Branch and Bound Algorithm for Global optimization , 1997, J. Glob. Optim..

[71]  Yasuhiko Yokote.  Object-Oriented Programming Languages , 1990 .

[72]  Paul I. Barton,et al.  Generalized branch-and-cut framework for mixed-integer nonlinear optimization problems , 2000 .

[73]  石塚 陽,et al.  On Global Optimality Conditions for D.C. Programming Problems , 1994 .

[74]  Panos M. Pardalos,et al.  Parallel search algorithms in global optimization , 1989 .

[75]  Hanif D. Sherali,et al.  A new reformulation-linearization technique for bilinear programming problems , 1992, J. Glob. Optim..

[76]  Ignacio E. Grossmann,et al.  A Branch and Contract Algorithm for Problems with Concave Univariate, Bilinear and Linear Fractional Terms , 1999, J. Glob. Optim..

[77]  David Q. Mayne,et al.  Efficient domain partitioning algorithms for global optimization of rational and Lipschitz continuous functions , 1989 .

[78]  Christodoulos A. Floudas,et al.  New properties and computational improvement of the GOP algorithm for problems with quadratic objective functions and constraints , 1993, J. Glob. Optim..

[79]  K. Grasse,et al.  A general class of branch-and-bound methods in global optimization with some new approaches for concave minimization , 1986 .

[80]  Immanuel M. Bomze,et al.  On Standard Quadratic Optimization Problems , 1998, J. Glob. Optim..

[81]  Edward M. B. Smith,et al.  On the optimal design of continuous processes , 1996 .

[82]  Duan Li,et al.  Convexification and existence of a saddle point in a pth-power reformulation for nonconvex constrained optimization☆ , 2001 .

[83]  Warren P. Adams,et al.  A Tight Linearization and an Algorithm for Zero-One Quadratic Programming Problems , 1986 .

[84]  Edward M. B. Smith,et al.  Global optimisation of nonconvex MINLPs , 1997 .

[85]  Garth P. McCormick,et al.  Computability of global solutions to factorable nonconvex programs: Part I — Convex underestimating problems , 1976, Math. Program..

[86]  Alfred V. Aho,et al.  Data Structures and Algorithms , 1983 .

[87]  Christodoulos A. Floudas,et al.  αBB: A global optimization method for general constrained nonconvex problems , 1995, J. Glob. Optim..

[88]  Hanif D. Sherali,et al.  Global Optimization of Nonconvex Polynomial Programming Problems Having Rational Exponents , 1998, J. Glob. Optim..

[89]  Lars Svensson,et al.  Computing global minima to polynomial optimization problems using Gröbner bases , 1995, J. Glob. Optim..

[90]  Anthony V. Fiacco,et al.  Nonlinear programming;: Sequential unconstrained minimization techniques , 1968 .

[91]  Henry Margenau,et al.  The mathematics of physics and chemistry , 1943 .

[92]  Mahmoud M. El-Halwagi,et al.  Global Optimization of Nonconvex MINLP’s by Interval Analysis , 1996 .

[93]  W. Murray,et al.  ALGORITHMS FOR GLOBAL OPTIMIZATION AND DISCRETE PROBLEMS BASED ON METHODS FOR LOCAL OPTIMIZATION , 2002 .

[94]  John E. Beasley,et al.  Heuristic algorithms for the unconstrained binary quadratic programming problem , 1998 .

[95]  A. S. Strekalovsky.  Extremal problems with D.C.-constraints , 2001 .

[96]  Hanif D. Sherali,et al.  On Finitely Terminating Branch-and-Bound Algorithms for Some Global Optimization Problems , 1999, SIAM J. Optim..

[97]  Marco Dorigo,et al.  Distributed Optimization by Ant Colonies , 1992 .

[98]  Antonio Frangioni.  On a new class of bilevel programming problems and its use for reformulating mixed integer problems , 1995 .

[99]  A. ilinskas.  Axiomatic characterization of a global optimization algorithm and investigation of its search strategy , 1985 .

[100]  Nikolaos V. Sahinidis,et al.  Convex extensions and envelopes of lower semi-continuous functions , 2002, Math. Program..

[101]  David Kendrick,et al.  GAMS, a user's guide , 1988, SGNM.

[102]  V. K. Jayaraman,et al.  Ant Colony Approach to Continuous Function Optimization , 2000 .

[103]  P. Pardalos.  Enumerative techniques for solving some nonconvex global optimization problems , 1988 .

[104]  Nikolaos V. Sahinidis,et al.  A Finite Algorithm for Global Minimization of Separable Concave Programs , 1998, J. Glob. Optim..

[105]  Christodoulos A. Floudas,et al.  Mixed-Integer Nonlinear Optimization in Process Synthesis , 1998 .

[106]  Leo Liberti,et al.  Convex Envelopes of Monomials of Odd Degree , 2003, J. Glob. Optim..

[107]  T. Westerlund,et al.  Solving Pseudo-Convex Mixed Integer Optimization Problems by Cutting Plane Techniques , 2002 .

[108]  Panos M. Pardalos,et al.  Constrained Global Optimization: Algorithms and Applications , 1987, Lecture Notes in Computer Science.

[109]  James E. Falk,et al.  Jointly Constrained Biconvex Programming , 1983, Math. Oper. Res..

[110]  Bjarne Stroustrup,et al.  C++ Programming Language , 1986, IEEE Softw..

[111]  Christodoulos A. Floudas,et al.  Global optimization of MINLP problems in Process Synthesis and Design , 1997 .

[112]  Masayuki Hirafuji,et al.  A global optimization algorithm based on the process of evolution in complex biological systems , 2000 .

[113]  K. Ciesielski.  Set Theory for the Working Mathematician , 1997 .

[114]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms , 1993 .

[115]  Roman G. Strongin,et al.  Global optimization with non-convex constraints , 2000 .

[116]  Aharon Ben-Tal,et al.  Global minimization by reducing the duality gap , 1994, Math. Program..

[117]  Christodoulos A. Floudas,et al.  Rigorous convex underestimators for general twice-differentiable problems , 1996, J. Glob. Optim..

[118]  C. Adjiman,et al.  A global optimization method, αBB, for general twice-differentiable constrained NLPs—II. Implementation and computational results , 1998 .

[119]  Eldon Hansen,et al.  Global optimization using interval analysis , 1992, Pure and applied mathematics.

[120]  L. Foulds,et al.  A bilinear approach to the pooling problem , 1992 .

[121]  Warren P. Adams,et al.  A Reformulation-Linearization Technique for Solving Discrete and Continuous Nonconvex Problems , 1998 .

[122]  ScienceDirect.  Comptes rendus de l'Académie des sciences. Série I, Mathématique , 2001 .

[123]  N. Sahinidis,et al.  Global optimization of nonconvex NLPs and MINLPs with applications in process design , 1995 .

[124]  David A. Cox,et al.  Ideals, Varieties, and Algorithms , 1997 .

[125]  J. Rokne,et al.  Efficiency of a Global Optimization Algorithm , 1987 .

[126]  G. Dueck,et al.  Record Breaking Optimization Results Using the Ruin and Recreate Principle , 2000 .

[127]  Bruce J. Berne,et al.  Global Optimization : Quantum Thermal Annealing with Path Integral Monte Carlo , 1999 .

[128]  William J. Cook,et al.  Combinatorial optimization , 1997 .

[129]  R. Hoprst.  Deterministic global optimization with partition sets whose feasibility is not known: application to concave minimization reverse convex , 1988 .

[130]  R. Horst,et al.  Global Optimization: Deterministic Approaches , 1992 .

[131]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[132]  Hanif D. Sherali,et al.  Global optimization of nonconvex factorable programming problems , 2001, Math. Program..