Convex nondifferentiable optimization: A survey focused on the analytic center cutting plane method

We present a survey of nondifferentiable optimization problems and methods with special focus on the analytic center cutting plane method. We propose a self-contained convergence analysis that uses the formalism of the theory of self-concordant functions, but for the main results, we give direct proofs based on the properties of the logarithmic function. We also provide an in-depth analysis of two extensions that are very relevant to practical problems: the case of multiple cuts and the case of deep cuts. We further examine extensions to problems including feasible sets partially described by an explicit barrier function, and to the case of nonlinear cuts. Finally, we review several implementation issues and discuss some applications.

[1]  J. Shapiro A Survey of Lagrangian Techniques for Discrete Optimization. , 1979 .

[2]  Olivier Péton The homogeneous analytic center cutting plane method , 2002 .

[3]  Alain Haurie,et al.  Advanced mathematical programming modeling to assess the benefits from international CO2 abatement cooperation , 1997 .

[4]  Tamás Terlaky,et al.  A logarithmic barrier cutting plane method for convex programming , 1995, Ann. Oper. Res..

[5]  Philippe Mahey,et al.  A Survey of Algorithms for Convex Multicommodity Flow Problems , 2000 .

[6]  Yurii Nesterov,et al.  New variants of bundle methods , 1995, Math. Program..

[7]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[8]  Zhi-Quan Luo,et al.  Interior point least squares estimation: transient convergence analysis and application to MMSE decision-feedback equalization , 2001, IEEE Trans. Signal Process..

[9]  Jean-Louis Goffin,et al.  A Nonlinear Analytic Center Cutting Plane Method for a Class of Convex Programming Problems , 1996, SIAM J. Optim..

[10]  Samir Elhedhli,et al.  The integration of an interior-point cutting plane method within a branch-and-price algorithm , 2004, Math. Program..

[11]  O. P etony,et al.  Homogeneous Analytic Center Cutting Plane Methods with Approximate Centers , 1999 .

[12]  Jacek Gondzio,et al.  A Structure-Exploiting Tool in Algebraic Modeling Languages , 2000 .

[13]  George L. Nemhauser,et al.  Handbooks in operations research and management science , 1989 .

[14]  R. Gomory,et al.  A Linear Programming Approach to the Cutting-Stock Problem , 1961 .

[15]  Yurii Nesterov,et al.  Complexity estimates of some cutting plane methods based on the analytic barrier , 1995, Math. Program..

[16]  Marshall L. Fisher,et al.  The Lagrangian Relaxation Method for Solving Integer Programming Problems , 2004, Manag. Sci..

[17]  Ralph E. Gomory,et al.  A Linear Programming Approach to the Cutting Stock Problem---Part II , 1963 .

[18]  John E. Mitchell,et al.  Computational Experience with an Interior Point Cutting Plane Algorithm , 1999, SIAM J. Optim..

[19]  Jean-Louis Goffin,et al.  On a Primal-Dual Analytic Center Cutting Plane Method for Variational Inequalities , 1997, Comput. Optim. Appl..

[20]  Carl Vogt A Structure-Exploiting Tool in Algebraic Modeling Languages , 2000 .

[21]  Larry J. LeBlanc,et al.  A new proximal decomposition algorithm for routing in telecommunication networks , 1998 .

[22]  Jean-Philippe Vial,et al.  Multiple Cuts in the Analytic Center Cutting Plane Method , 1998, SIAM J. Optim..

[23]  Yinyu Ye,et al.  A Short-Cut Potential Reduction Algorithm for Linear Programming , 1993 .

[24]  Yinyu Ye,et al.  Complexity analysis of the analytic center cutting plane method that uses multiple cuts , 1997, Math. Program..

[25]  Jean-Philippe Vial,et al.  A two-cut approach in the analytic center cutting plane method , 1997, Math. Methods Oper. Res..

[26]  S. Graves Using Lagrangean Techniques to Solve Hierarchical Production Planning Problems , 1982 .

[27]  Yinyu Ye,et al.  Complexity Analysis of an Interior Cutting Plane Method for Convex Feasibility Problems , 1996, SIAM J. Optim..

[28]  Jacek Gondzio,et al.  Solving nonlinear multicommodity flow problems by the analytic center cutting plane method , 1997, Math. Program..

[29]  G. Sonnevend New Algorithms in Convex Programming Based on a Notion of “Centre” (for Systems of Analytic Inequalities) and on Rational Extrapolation , 1988 .

[30]  C. Lemaréchal Nondifferentiable optimization , 1989 .

[31]  Jean-Philippe Vial,et al.  Theory and algorithms for linear optimization - an interior point approach , 1998, Wiley-Interscience series in discrete mathematics and optimization.

[32]  Hans-Jakob Lüthi,et al.  The Analytic Center Quadratic Cut Method for Strongly Monotone Variational Inequality Problems , 2000, SIAM J. Optim..

[33]  Michael J. Todd,et al.  Solving combinatorial optimization problems using Karmarkar's algorithm , 1992, Math. Program..

[34]  J. Goffin,et al.  A Lagrangian Relaxation of the Capacitated Multi-Item Lot Sizing Problem Solved with an Interior Poi , 1997 .

[35]  Yurii Nesterov,et al.  Homogeneous Analytic Center Cutting Plane Methods for Convex Problems and Variational Inequalities , 1999, SIAM J. Optim..

[36]  N. Shor Nondifferentiable Optimization and Polynomial Problems , 1998 .

[37]  Srini Ramaswamy,et al.  A Long-Step, Cutting Plane Algorithm for Linear and Convex Programming , 2000, Ann. Oper. Res..

[38]  Philip Wolfe,et al.  Validation of subgradient optimization , 1974, Math. Program..

[39]  J. Goffin,et al.  Decomposition and nondifferentiable optimization with the projective algorithm , 1992 .

[40]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[41]  Yinyu Ye,et al.  Interior point algorithms: theory and analysis , 1997 .

[42]  Jie Sun,et al.  An Analytic Center Based Column Generation Algorithm for Convex Quadratic Feasibility Problems , 1998, SIAM J. Optim..

[43]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[44]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[45]  Jean-Louis Goffin,et al.  An analytic center quadratic cut method for the convex quadratic feasibility problem , 2002, Math. Program..

[46]  Jacek Gondzio,et al.  A PLanning Model with One Million Scenarios Solved on an Affordable Parallel Machine , 1998 .

[47]  Boris Polyak Minimization of unsmooth functionals , 1969 .

[48]  G. Dantzig,et al.  The decomposition algorithm for linear programming: notes on linear programming and extensions-part 57. , 1961 .

[49]  Jacek Gondzio,et al.  Warm start of the primal-dual method applied in the cutting-plane scheme , 1998, Math. Program..

[50]  Jean-Philippe Vial,et al.  On the computation of weighted analytic centers and dual ellipsoids with the projective algorithm , 1993, Math. Program..

[51]  J. E. Kelley,et al.  The Cutting-Plane Method for Solving Convex Programs , 1960 .

[52]  Jean-Philippe Vial,et al.  Shallow, deep and very deep cuts in the analytic center cutting plane method , 1999, Math. Program..

[53]  A. Auslender Optimisation : méthodes numériques , 1976 .

[54]  J. Mitchell,et al.  Solving Linear Ordering Problems with a Combined Interior Point/Simplex Cutting Plane Algorithm , 2000 .

[55]  G. Dantzig,et al.  THE DECOMPOSITION ALGORITHM FOR LINEAR PROGRAMS , 1961 .

[56]  John E. Mitchell,et al.  A Homogenized Cutting Plane Method to Solve the Convex Feasibility Problem , 2001 .

[57]  K. Kiwiel Methods of Descent for Nondifferentiable Optimization , 1985 .

[58]  E. Yaz Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.

[59]  P. Wolfe Note on a method of conjugate subgradients for minimizing nondifferentiable functions , 1974 .

[60]  Pravin M. Vaidya,et al.  A new algorithm for minimizing convex functions over convex sets , 1996, Math. Program..

[61]  Leon S. Lasdon,et al.  Optimization Theory of Large Systems , 1970 .

[62]  Luigi Fratta,et al.  The flow deviation method: An approach to store-and-forward communication network design , 1973, Networks.

[63]  V. Quintana,et al.  An interior-point/cutting-plane method to solve unit commitment problems , 1999 .

[64]  Jean-Philippe Vial,et al.  On Improvements to the Analytic Center Cutting Plane Method , 1998, Comput. Optim. Appl..

[65]  Philip Wolfe,et al.  Note on a method of conjugate subgradients for minimizing nondifferentiable functions , 1974, Math. Program..

[66]  Hans-Jakob L Uthi,et al.  The Analytic Center Quadratic Cut Method (acqcm) for Strongly Monotone Variational Inequality Problems , 1998 .

[67]  Er-Wei Bai,et al.  Bounded error parameter estimation: a sequential analytic center approach , 1999, IEEE Trans. Autom. Control..

[68]  Stephen P. Boyd,et al.  Linear Matrix Inequalities in Systems and Control Theory , 1994 .

[69]  Krzysztof C. Kiwiel,et al.  Efficiency of the Analytic Center Cutting Plane Method for Convex Minimization , 1997, SIAM J. Optim..

[70]  Pierre Hansen,et al.  An Interior Point Algorithm for Minimum Sum-of-Squares Clustering , 1997, SIAM J. Sci. Comput..

[71]  Dimitri P. Bertsekas,et al.  Data Networks , 1986 .

[72]  A. A. Goldstein,et al.  Newton's method for convex programming and Tchebycheff approximation , 1959, Numerische Mathematik.

[73]  Richard M. Karp,et al.  The Traveling-Salesman Problem and Minimum Spanning Trees , 1970, Oper. Res..

[74]  Jean-Philippe Vial,et al.  A cutting plane method from analytic centers for stochastic programming , 1995, Math. Program..

[75]  Jack Elzinga,et al.  A central cutting plane algorithm for the convex programming problem , 1975, Math. Program..

[76]  Q. Zhang A NEW POLYNOMIAL-TIME ALGORITHM FOR LP , 1996 .

[77]  J. Gondizo,et al.  Warm Start and ε-Subgradients in a Cutting Plane Scheme for Block-Angular Linear Programs , 1999, Comput. Optim. Appl..

[78]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[79]  Jacek Gondzio,et al.  ACCPM — A library for convex optimization based on an analytic center cutting plane method☆ , 1996 .

[80]  Stephen P. Boyd,et al.  Linear controller design: limits of performance , 1991 .

[81]  Robert R. Meyer,et al.  Multicoordination Methods for Solving Convex Block-Angular Programs , 1999, SIAM J. Optim..

[82]  Jean-Louis Goffin,et al.  Variational Inequalities With Quadratic Cuts , 1998 .

[83]  R. Wets,et al.  L-SHAPED LINEAR PROGRAMS WITH APPLICATIONS TO OPTIMAL CONTROL AND STOCHASTIC PROGRAMMING. , 1969 .

[84]  Larry J. LeBlanc,et al.  A new proximal decomposition algorithm for routing in telecommunication networks , 1998, Networks.

[85]  Richard M. Karp,et al.  The traveling-salesman problem and minimum spanning trees: Part II , 1971, Math. Program..

[86]  George B. Dantzig,et al.  Decomposition Principle for Linear Programs , 1960 .

[87]  Abdel Lisser,et al.  Optimal Joint Synthesis of Base and Reserve Telecommunication Networks , 1995 .

[88]  J. Gondzio,et al.  Using an interior point method for the master problem in a decomposition approach , 1997 .

[89]  M. Shirosaki Another proof of the defect relation for moving targets , 1991 .

[90]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, Comb..

[91]  Krzysztof C. Kiwiel,et al.  A note on some analytic center cutting plane methods for convex feasibility and minimization problems , 1996, Comput. Optim. Appl..

[92]  Zhi-Quan Luo,et al.  Adaptive linear filtering using interior point optimization techniques , 2000, IEEE Trans. Signal Process..

[93]  Yinyu Ye,et al.  A Potential Reduction Algorithm Allowing Column Generation , 1992, SIAM J. Optim..

[94]  John L. Nazareth,et al.  The decomposition principle and algorithms for linear programming , 1991 .

[95]  Jean-Philippe Vial,et al.  Multiple Cuts with a Homogeneous Analytic Center Cutting Plane Method , 2003, Comput. Optim. Appl..

[96]  Warren B. Powell,et al.  Multicommodity network flows: The impact of formulation on decomposition , 1993, Math. Program..

[97]  Leonid Khachiyan,et al.  On the complexity of approximating the maximal inscribed ellipsoid for a polytope , 1993, Math. Program..

[98]  Jie Sun,et al.  A Polynomial Cutting Surfaces Algorithm for the Convex Feasibility Problem Defined by Self-Concordant Inequalities , 2000, Comput. Optim. Appl..

[99]  P. Wolfe,et al.  A METHOD OF CONJUGATE SUBGRADIENTS FOR , 1975 .

[100]  Robert Sarkissian Telecommunications networks routing and survivability optimization using a central cutting plane method , 1998 .

[101]  Kurt M. Anstreicher,et al.  Towards a Practical Volumetric Cutting Plane Method for Convex Programming , 1998, SIAM J. Optim..

[102]  Zhi-Quan Luo,et al.  Complexity analysis of logarithmic barrier decomposition methods for semi-infinite linear programming , 1999 .