Continuous methods for convex programming and convex semidefinite programming

In this thesis, we study several interior point continuous trajectories for linearly constrained convex programming (CP) and convex semidefinite programming (SDP). The continuous trajectories are characterized as the solution trajectories of corresponding ordinary differential equation (ODE) systems. All our ODE systems are closely related to interior point methods. First, we propose and analyze three continuous trajectories, which are the solutions of three ODE systems for linearly constrained convex programming. The three ODE systems are formulated based on an variant of the affine scaling direction, the central path, and the affine scaling direction in interior point methods. The resulting solutions of the first two ODE systems are called generalized affine scaling trajectory and generalized central path, respectively. Under some mild conditions, the properties of the continuous trajectories, the optimality and convergence of the continuous trajectories are all obtained. Furthermore, we show that for the example of Gilbert et al. [Math. Program., 103, 63-94 (2005)], where the central path does not converge, our generalized central path converges to an optimal solution of the same example in the limit. Then we analyze two primal dual continuous trajectories for convex programming. The two continuous trajectories are derived from the primal-dual path-following method and the primal-dual affine scaling method, respectively. Theoretical properties of the two interior point continuous trajectories are fully studied. The optimality and convergence of both interior point continuous trajectories are obtained for any interior feasible point under some mild conditions. In particular, with proper choice of some parameters, the convergence for both continuous trajectories does not require the strict complementarity or the analyticity of the objective function. For convex semidefinite programming, four interior continuous trajectories defined by matrix differential equations are proposed and analyzed. Optimality and convergence of the continuous trajectories are also obtained under some mild conditions.

[1]  Zhi-Quan Luo,et al.  Superlinear Convergence of a Symmetric Primal-Dual Path Following Algorithm for Semidefinite Programming , 1998, SIAM J. Optim..

[2]  C. Roos,et al.  New trajectory-following polynomial-time algorithm for linear programming problems , 1989 .

[3]  Takashi Tsuchiya,et al.  Global Convergence of the Affine Scaling Algorithm for Convex Quadratic Programming , 1998, SIAM J. Optim..

[4]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[5]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[6]  D. Bayer,et al.  The nonlinear geometry of linear programming. II. Legendre transform coordinates and central trajectories , 1989 .

[7]  Etienne de Klerk,et al.  On the Convergence of the Central Path in Semidefinite Optimization , 2002, SIAM J. Optim..

[8]  Renato D. C. Monteiro,et al.  Primal-Dual Path-Following Algorithms for Semidefinite Programming , 1997, SIAM J. Optim..

[9]  Jun Wang,et al.  A recurrent neural network for real-time semidefinite programming , 1999, IEEE Trans. Neural Networks.

[10]  Clóvis C. Gonzaga,et al.  Examples of ill-behaved central paths in convex optimization , 2005, Math. Program..

[11]  L. M. Graña Drummond,et al.  THE CENTRAL PATH IN SMOOTH CONVEX SEMIDEFINITE PROGRAMS , 2002 .

[13]  E. Akin,et al.  Dynamics of games and genes: Discrete versus continuous time , 1983 .

[14]  Josef Stoer,et al.  Analysis of infeasible-interior-point paths arising with semidefinite linear complementarity problems , 2004, Math. Program..

[15]  Jie Sun,et al.  A convergence proof for an affine-scaling algorithm for convex quadratic programming without nondegeneracy assumptions , 1993, Math. Program..

[16]  Li-Zhi Liao,et al.  A Study of the Dual Affine Scaling Continuous Trajectories for Linear Programming , 2014, J. Optim. Theory Appl..

[17]  Renato D. C. Monteiro,et al.  Interior path following primal-dual algorithms. part I: Linear programming , 1989, Math. Program..

[18]  Chee-Khian Sim,et al.  Asymptotic Behavior of HKM Paths in Interior Point Methods for Monotone Semidefinite Linear Complementarity Problems: General Theory , 2005 .

[19]  丸山 徹 Convex Analysisの二,三の進展について , 1977 .

[20]  M. Todd,et al.  Inexact primal-dual path-following algorithms for a special class of convex quadratic SDP and related problems , 2005 .

[21]  P. Boggs,et al.  On the convergence behavior of trajectories for linear programming , 1988 .

[22]  Leonid Faybusovich,et al.  Dikin's algorithm for matrix linear programming problems , 1993, System Modelling and Optimization.

[23]  Paul Tseng,et al.  On the convergence of the affine-scaling algorithm , 1992, Math. Program..

[24]  Michael J. Todd,et al.  A Centered Projective Algorithm for Linear Programming , 1990, Math. Oper. Res..

[25]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[26]  Takashi Tsuchiya,et al.  Global Convergence Property of the Affine Scaling Methods for Primal Degenerate Linear Programming Problems , 1992, Math. Oper. Res..

[27]  Nimrod Megiddo,et al.  Boundary Behavior of Interior Point Algorithms in Linear Programming , 1989, Math. Oper. Res..

[28]  Jishan Zhu,et al.  A path following algorithm for a class of convex programming problems , 1992, ZOR Methods Model. Oper. Res..

[29]  M. J. D. Powell,et al.  Nonlinear Programming—Sequential Unconstrained Minimization Techniques , 1969 .

[30]  Yinyu Ye,et al.  A primal-dual interior point method whose running time depends only on the constraint matrix , 1996, Math. Program..

[31]  Katya Scheinberg,et al.  Interior Point Trajectories in Semidefinite Programming , 1998, SIAM J. Optim..

[32]  Osman Güler,et al.  Limiting behavior of weighted central paths in linear programming , 1994, Math. Program..

[33]  AN ANALOGUE OF MOREAU ' S PROXIMATION THEOREM , WITH APPLICATION TO THE NONLINEAR COMPLEMENTARITY PROBLEM , 2012 .

[34]  Hiroshi Yamashita,et al.  A primal–dual interior point method for nonlinear semidefinite programming , 2011, Mathematical Programming.

[35]  James Renegar,et al.  A polynomial-time algorithm, based on Newton's method, for linear programming , 1988, Math. Program..

[36]  A. I. Perov,et al.  On the convergence of an iterative process , 1977 .

[37]  Ya-Xiang Yuan,et al.  A Predictor–Corrector Algorithm for QSDP Combining Dikin-Type and Newton Centering Steps , 2001, Ann. Oper. Res..

[38]  Hiroshi Yamashita,et al.  Local and superlinear convergence of a primal-dual interior point method for nonlinear semidefinite programming , 2012, Math. Program..

[39]  Kees Roos,et al.  A survey of search directions in interior point methods for linear programming , 1991, Math. Program..

[40]  Mauricio G. C. Resende,et al.  A Polynomial-Time Primal-Dual Affine Scaling Algorithm for Linear and Convex Quadratic Programming and Its Power Series Extension , 1990, Math. Oper. Res..

[41]  Harold R. Parks,et al.  A Primer of Real Analytic Functions , 1992 .

[42]  R. C. Monteiro,et al.  Interior path following primal-dual algorithms , 1988 .

[43]  Yin Zhang,et al.  A unified analysis for a class of long-step primal-dual path-following interior-point algorithms for semidefinite programming , 1998, Math. Program..

[44]  N. Megiddo Pathways to the optimal set in linear programming , 1989 .

[45]  Takashi Tsuchiya,et al.  Global convergence of the affine scaling methods for degenerate linear programming problems , 1991, Math. Program..

[46]  Benar Fux Svaiter,et al.  On Well Definedness of the Central Path , 1999 .

[47]  Renato D. C. Monteiro,et al.  On the Continuous Trajectories for a Potential Reduction Algorithm for Linear Programming , 1992, Math. Oper. Res..

[48]  Romesh Saigal,et al.  A simple proof of a primal affine scaling method , 1996, Ann. Oper. Res..

[49]  Chee-Khian Sim,et al.  Underlying paths in interior point methods for the monotone semidefinite linear complementarity problem , 2007, Math. Program..

[50]  Moody T. Chu,et al.  Dynamical System Characterization of the Central Path and Its Variants - A Revisit , 2011, SIAM J. Appl. Dyn. Syst..

[51]  M. Kojima,et al.  A primal-dual interior point algorithm for linear programming , 1988 .

[52]  A. Iusem,et al.  Central Paths, Generalized Proximal Point Methods, and Cauchy Trajectories in Riemannian Manifolds , 1999 .

[53]  Masakazu Muramatsu Affine scaling algorithm fails for semidefinite programming , 1998, Math. Program..

[54]  Kim-Chuan Toh,et al.  On the Nesterov-Todd Direction in Semidefinite Programming , 1998, SIAM J. Optim..

[55]  Yinyu Ye,et al.  An extension of Karmarkar's projective algorithm for convex quadratic programming , 1989, Math. Program..

[56]  Renato D. C. Monteiro,et al.  On the Existence and Convergence of the Central Path for Convex Programming and Some Duality Results , 1998, Comput. Optim. Appl..

[57]  Jie Sun A convergence analysis for a convex version of Dikin's algorithm , 1996, Ann. Oper. Res..

[58]  Renato D. C. Monteiro A globally convergent primal—dual interior point algorithm for convex programming , 1994, Math. Program..

[59]  Earl R. Barnes,et al.  A variation on Karmarkar’s algorithm for solving linear programming problems , 1986, Math. Program..

[60]  Héctor Ramírez Cabrera,et al.  On the Central Paths and Cauchy Trajectories in Semidefinite Programming , 2010, Kybernetika.

[61]  R. Monteiro,et al.  Limiting behavior of the derivatives of certain trajectories associated with a monotone horizontal linear complementarity problem , 1996 .

[62]  Shuzhong Zhang,et al.  Symmetric primal-dual path-following algorithms for semidefinite programming , 1999 .

[63]  Nicholas J. Higham,et al.  Functions of matrices - theory and computation , 2008 .

[64]  Clóvis C. Gonzaga,et al.  Path-Following Methods for Linear Programming , 1992, SIAM Rev..

[65]  Michael L. Overton,et al.  Primal-Dual Interior-Point Methods for Semidefinite Programming: Convergence Rates, Stability and Numerical Results , 1998, SIAM J. Optim..

[66]  Yin Zhang,et al.  On Extending Some Primal-Dual Interior-Point Algorithms From Linear Programming to Semidefinite Programming , 1998, SIAM J. Optim..

[67]  Paul Tseng,et al.  A first-order interior-point method for linearly constrained smooth optimization , 2011, Math. Program..

[68]  Charles Pugh,et al.  Functions of a Real Variable , 2002 .

[69]  Renato D. C. Monteiro,et al.  Limiting behavior of the affine scaling continuous trajectories for linear programming problems , 1991, Math. Program..

[70]  Margaréta Halická,et al.  Limiting behaviour and analyticity of weighted central paths in semidefinite programming , 2010, Optim. Methods Softw..

[71]  Alfredo N. Iusem,et al.  On the central path for nonlinear semidefinite programming , 2000, RAIRO Oper. Res..

[72]  Kim-Chuan Toh,et al.  An inexact primal–dual path following algorithm for convex quadratic SDP , 2007, Math. Program..

[73]  R. Saigal,et al.  Handbook of semidefinite programming : theory, algorithms, and applications , 2000 .

[74]  Anton van den Hengel,et al.  Semidefinite Programming , 2014, Computer Vision, A Reference Guide.

[75]  M. Fiedler,et al.  Matrix Inequalities , 1966 .

[76]  Robert J. Vanderbei,et al.  A modification of karmarkar's linear programming algorithm , 1986, Algorithmica.

[77]  Weiping Li,et al.  Applied Nonlinear Control , 1991 .

[78]  Renato D. C. Monteiro,et al.  Limiting behavior of the Alizadeh–Haeberly–Overton weighted paths in semidefinite programming , 2007, Optim. Methods Softw..