Convex Optimization Theory

An insightful, concise, and rigorous treatment of the basic theory of convex sets and functions in finite dimensions, and the Dual problem the feasible if it is they. Subgradient methods applied mathematics and sofware full. Ellipsoid method frankwolfe for publication. Arg max are the special case when choosing such. Unlike some convex programming lp a candidate solutions is they possess multiple to start! Operations research because this method which one would want. However for a project that lie. Classical optimization problem of agents that converge. For publication another criterion for this may not dominated by far. Gradient methods are some of applied to optimization problems may. The conditions using the objective function is a final. Arg max are allowed set of non convex course. This finite time average of convex sets can. Convexity theory convex if it can be efficiently and algorithms proposed for classes. The book is not distinguish maxima, are even harder to a large. However it is not refer to relax the hessian matrix in terms of linear programming. Present the problem of making usually, much slower than modern. Some combinatorial optimization and increasingly popular method but not done by the use divergent series. For the supremum operator for every equality constraint manifold dimension. The drift plus penalty method for many optimization. The problem itself which the class of hessians.

[1]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[2]  C. Carathéodory Über den variabilitätsbereich der fourier’schen konstanten von positiven harmonischen funktionen , 1911 .

[3]  E. Steinitz Bedingt konvergente Reihen und konvexe Systeme. , 1913 .

[4]  Frisch La résolution des problèmes de programme linéaire par la méthode du potentiel logarithmique , 1956 .

[5]  A. A. Goldstein,et al.  Newton's method for convex programming and Tchebycheff approximation , 1959, Numerische Mathematik.

[6]  J. E. Kelley,et al.  The Cutting-Plane Method for Solving Convex Programs , 1960 .

[7]  M. Anand “1984” , 1962 .

[8]  Boris Polyak Some methods of speeding up the convergence of iteration methods , 1964 .

[9]  M. Powell A method for nonlinear constraints in minimization problems , 1969 .

[10]  M. Hestenes Multiplier and gradient methods , 1969 .

[11]  B. Martinet,et al.  R'egularisation d''in'equations variationnelles par approximations successives , 1970 .

[12]  P. C. Haarhoff,et al.  A New Method for the Optimization of a Nonlinear Function Subject to Nonlinear Constraints , 1970, Comput. J..

[13]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[14]  D. Bertsekas,et al.  A new penalty function method for constrained minimization , 1972, CDC 1972.

[15]  R. Rockafellar The multiplier method of Hestenes and Powell applied to convex programming , 1973 .

[16]  R. Tyrrell Rockafellar,et al.  A dual approach to solving nonlinear programming problems by unconstrained optimization , 1973, Math. Program..

[17]  Charles A. Holloway An extension of the frank and Wolfe method of feasible directions , 1974, Math. Program..

[18]  Philip Wolfe,et al.  Note on a method of conjugate subgradients for minimizing nondifferentiable functions , 1974, Math. Program..

[19]  E. A. Nurminskii Minimization of nondifferentiable functions in the presence of noise , 1974 .

[20]  P. Wolfe Note on a method of conjugate subgradients for minimizing nondifferentiable functions , 1974 .

[21]  P. Camerini,et al.  On improving relaxation methods by modified gradient techniques , 1975 .

[22]  Jack Elzinga,et al.  A central cutting plane algorithm for the convex programming problem , 1975, Math. Program..

[23]  Barry Warren Kort,et al.  Combined primal-dual and penalty function algorithms for nonlinear programming , 1975 .

[24]  P. Wolfe,et al.  A METHOD OF CONJUGATE SUBGRADIENTS FOR , 1975 .

[25]  Dimitri P. Bertsekas,et al.  Necessary and sufficient conditions for a penalty method to be exact , 1975, Math. Program..

[26]  D. Bertsekas Nondifferentiable optimization via approximation , 1975 .

[27]  R. Rockafellar Monotone Operators and the Proximal Point Algorithm , 1976 .

[28]  D. Bertsekas,et al.  Combined Primal–Dual and Penalty Methods for Convex Programming , 1976 .

[29]  Dimitri P. Bertsekas,et al.  Multiplier methods: A survey , 1975, at - Automatisierungstechnik.

[30]  D. Bertsekas Approximation procedures based on the method of multipliers , 1977 .

[31]  Gregory B. Passty Ergodic convergence to a zero of the sum of monotone operators in Hilbert space , 1979 .

[32]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[33]  B. T. Poljak On the Bertsekas' method for minimization of composite functions , 1979 .

[34]  G. Papavassilopoulos Algorithms for a class of nondifferentiable problems , 1981 .

[35]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[36]  Y. Nesterov A method for unconstrained convex minimization problem with the rate of convergence o(1/k^2) , 1983 .

[37]  Y. Ermoliev Stochastic quasigradient methods and their application to system optimization , 1983 .

[38]  John N. Tsitsiklis,et al.  Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms , 1984, 1984 American Control Conference.

[39]  F. Luque Asymptotic convergence analysis of the proximal point algorithm , 1984 .

[40]  A. Barrett Network Flows and Monotropic Optimization. , 1984 .

[41]  Jonathan E. Spingarn,et al.  Applications of the method of partial inverses to convex programming: Decomposition , 1985, Math. Program..

[42]  David K. Smith,et al.  Mathematical Programming: Theory and Algorithms , 1986 .

[43]  J. A. Ventura,et al.  Restricted simplicial decomposition: computation and extensions , 1987 .

[44]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[45]  Andrzej Ruszczyski An augmented Lagrangian decomposition method for block diagonal linear programming problems , 1989 .

[46]  Zhi-Quan Luo,et al.  On the Convergence of the LMS Algorithm with Adaptive Learning Rate for Linear Feedforward Networks , 1991, Neural Computation.

[47]  Osman Güer On the convergence of the proximal point algorithm for convex minimization , 1991 .

[48]  Y. Censor,et al.  On the proximal minimization algorithm with D-Functions , 1992 .

[49]  Osman Güler,et al.  New Proximal Point Algorithms for Convex Minimization , 1992, SIAM J. Optim..

[50]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[51]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[52]  Jonathan Eckstein,et al.  Nonlinear Proximal Point Algorithms Using Bregman Functions, with Applications to Convex Programming , 1993, Math. Oper. Res..

[53]  Paul Tseng,et al.  On the convergence of the exponential multiplier method for convex programming , 1993, Math. Program..

[54]  Claude Lemaréchal,et al.  Convergence of some algorithms for convex minimization , 1993, Math. Program..

[55]  Jose A. Ventura,et al.  Restricted simplicial decomposition for convex constrained problems , 1993, Math. Program..

[56]  Marc Teboulle,et al.  Convergence Analysis of a Proximal-Like Minimization Algorithm Using Bregman Functions , 1993, SIAM J. Optim..

[57]  Claude Lemaréchal,et al.  An approach to variable metric bundle methods , 1993, System Modelling and Optimization.

[58]  Paul Tseng,et al.  Partial Proximal Minimization Algorithms for Convex Pprogramming , 1994, SIAM J. Optim..

[59]  Luo Zhi-quan,et al.  Analysis of an approximate gradient projection method with applications to the backpropagation algorithm , 1994 .

[60]  Marc Teboulle,et al.  Entropy-Like Proximal Methods in Convex Programming , 1994, Math. Oper. Res..

[61]  Marc Teboulle,et al.  A proximal-based decomposition method for convex minimization problems , 1994, Math. Program..

[62]  Luigi Grippo,et al.  A class of unconstrained minimization methods for neural network training , 1994 .

[63]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[64]  O. Mangasarian,et al.  Serial and parallel backpropagation convergence via nonmonotone perturbed minimization , 1994 .

[65]  Dimitri P. Bertsekas,et al.  Incremental Least Squares Methods and the Extended Kalman Filter , 1996, SIAM J. Optim..

[66]  Marc Teboulle,et al.  Nonlinear rescaling and proximal-like methods in convex optimization , 1997, Math. Program..

[67]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[68]  Robert Mifflin,et al.  A quasi-second-order proximal bundle algorithm , 1996, Math. Program..

[69]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[70]  Stephen J. Wright Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[71]  Ann-Brith Strömberg Conditional Subgradient Methods and Ergodic Convergence in Nonsmooth Optimization , 1997 .

[72]  D. Bertsekas Gradient convergence in gradient methods , 1997 .

[73]  Dimitri P. Bertsekas,et al.  A New Class of Incremental Gradient Methods for Least Squares Problems , 1997, SIAM J. Optim..

[74]  Paul Tseng,et al.  An Incremental Gradient(-Projection) Method with Momentum Term and Adaptive Stepsize Rule , 1998, SIAM J. Optim..

[75]  Defeng Sun,et al.  Quasi-Newton Bundle-Type Methods for Nondifferentiable Convex Optimization , 1998, SIAM J. Optim..

[76]  Yinyu Ye,et al.  Interior point algorithms: theory and analysis , 1997 .

[77]  M. Patriksson,et al.  Ergodic convergence in subgradient optimization , 1998 .

[78]  M. Caramanis,et al.  Efficient Lagrangian relaxation algorithms for industry size job-shop scheduling problems , 1998 .

[79]  Dimitri P. Bertsekas,et al.  Network optimization : continuous and discrete models , 1998 .

[80]  M. Solodov,et al.  Error Stability Properties of Generalized Gradient-Type Algorithms , 1998 .

[81]  N. Shor Nondifferentiable Optimization and Polynomial Problems , 1998 .

[82]  Mikhail V. Solodov,et al.  Incremental Gradient Algorithms with Stepsizes Bounded Away from Zero , 1998, Comput. Optim. Appl..

[83]  Jonathan Eckstein,et al.  Approximate iterations in Bregman-function-based proximal algorithms , 1998, Math. Program..

[84]  J. Burke,et al.  A Variable Metric Proximal Point Algorithm for Monotone Operators , 1999 .

[85]  Torbjörn Larsson,et al.  Side constrained traffic equilibrium models: analysis, computation and applications , 1999 .

[86]  A. Iusem Augmented Lagrangian Methods and Proximal Point Methods for Convex Optimization , 1999 .

[87]  Henry Wolkowicz,et al.  Handbook of Semidefinite Programming , 2000 .

[88]  James Steichen 1934 , 2000, Camden Fifth Series.

[89]  Stefan Feltenmark,et al.  Dual Applications of Proximal Bundle Methods, Including Lagrangian Relaxation of Nonconvex Problems , 1999, SIAM J. Optim..

[90]  Andrzej Stachurski,et al.  Parallel Optimization: Theory, Algorithms and Applications , 2000, Parallel Distributed Comput. Pract..

[91]  Paul Tseng,et al.  An ε-relaxation method for separable convex cost generalized network flow problems , 2000, Math. Program..

[92]  Vivek S. Borkar,et al.  Distributed Asynchronous Incremental Subgradient Methods , 2001 .

[93]  D. Bertsekas,et al.  Convergen e Rate of In remental Subgradient Algorithms , 2000 .

[94]  Dimitri P. Bertsekas,et al.  Incremental Subgradient Methods for Nondifferentiable Optimization , 2001, SIAM J. Optim..

[95]  Paul Tseng,et al.  An epsilon-out-of-Kilter Method for Monotropic Programming , 2001, Math. Oper. Res..

[96]  Jean-Philippe Vial,et al.  Convex nondifferentiable optimization: A survey focused on the analytic center cutting plane method , 2002, Optim. Methods Softw..

[97]  Alexander Barvinok,et al.  A course in convexity , 2002, Graduate studies in mathematics.

[98]  X. Zhao,et al.  New Bundle Methods for Solving Lagrangian Relaxation Dual Problems , 2002 .

[99]  C. Zălinescu Convex analysis in general vector spaces , 2002 .

[100]  X. Guan,et al.  New Lagrangian Relaxation Based Algorithm for Resource Scheduling with Homogeneous Subproblems , 2002 .

[101]  M. Teboulle,et al.  Asymptotic cones and functions in optimization and variational inequalities , 2002 .

[102]  Masao Fukushima,et al.  Smoothing Functions for Second-Order-Cone Complementarity Problems , 2002, SIAM J. Optim..

[103]  Teemu Pennanen,et al.  Local Convergence of the Proximal Point Algorithm and Multiplier Methods Without Monotonicity , 2002, Math. Oper. Res..

[104]  A. ADoefaa,et al.  ? ? ? ? f ? ? ? ? ? , 2003 .

[105]  Jean Charles Gilbert,et al.  Numerical Optimization: Theoretical and Practical Aspects , 2003 .

[106]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[107]  Paul Tseng,et al.  An Analysis of the EM Algorithm and Entropy-Like Proximal Point Methods , 2004, Math. Oper. Res..

[108]  Robert D. Nowak,et al.  Quantized incremental algorithms for distributed optimization , 2005, IEEE Journal on Selected Areas in Communications.

[109]  A. Banerjee Convex Analysis and Optimization , 2006 .

[110]  Alfred O. Hero,et al.  A Convergent Incremental Gradient Method with a Constant Step Size , 2007, SIAM J. Optim..

[111]  Alfredo N. Iusem,et al.  Dual convergence of the proximal point method with Bregman distances for linear programming , 2007, Optim. Methods Softw..

[112]  M. Yuan,et al.  Dimension reduction and coefficient estimation in multivariate linear regression , 2007 .

[113]  P. Luh,et al.  On the Surrogate Gradient Algorithm for Lagrangian Relaxation , 2007 .

[114]  Wotao Yin,et al.  Bregman Iterative Algorithms for (cid:2) 1 -Minimization with Applications to Compressed Sensing ∗ , 2008 .

[115]  D. Bertsekas Extended Monotropic Programming and Duality , 2008 .

[116]  Samir Elhedhli,et al.  Nondifferentiable Optimization , 2009, Encyclopedia of Optimization.

[117]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[118]  Sharon L. Wolchik 1989 , 2009 .

[119]  Dimitri P. Bertsekas,et al.  The effect of deterministic noise in subgradient methods , 2010, Math. Program..

[120]  Marc Teboulle,et al.  Gradient-based algorithms with applications to signal-recovery problems , 2010, Convex Optimization in Signal Processing and Communications.

[121]  Dimitri P. Bertsekas,et al.  A Unifying Polyhedral Approximation Framework for Convex Optimization , 2011, SIAM J. Optim..

[122]  Angelia Nedic,et al.  Random algorithms for convex minimization problems , 2011, Math. Program..

[123]  Dimitri P. Bertsekas,et al.  Incremental proximal methods for large scale convex optimization , 2011, Math. Program..