Convex Optimization

Convex optimization problems arise frequently in many different fields. A comprehensive introduction to the subject, this book shows in detail how such problems can be solved numerically with great efficiency. The focus is on recognizing convex optimization problems and then finding the most appropriate technique for solving them. The text contains many worked examples and homework exercises and will appeal to students, researchers and practitioners in fields such as engineering, computer science, mathematics, statistics, finance, and economics.

[1]  J. Farkas.  Theorie der einfachen Ungleichungen. , 1902 .

[2]  J. Jensen.  Sur les fonctions convexes et les inégalités entre les valeurs moyennes , 1906 .

[3]  J. Jewkes,et al.  Theory of Location of Industries. , 1933 .

[4]  Karl Löwner.  Über monotone Matrixfunktionen , 1934 .

[5]  I. J. Schoenberg.  Remarks to Maurice Frechet's Article ``Sur La Definition Axiomatique D'Une Classe D'Espace Distances Vectoriellement Applicable Sur L'Espace De Hilbert , 1935 .

[6]  Th. Motzkin.  Beiträge zur Theorie der linearen Ungleichungen , 1936 .

[7]  Anton E. Mayer.  Theorie der konvexen Körper , 1936 .

[8]  J. Neumann.  A Model of General Economic Equilibrium , 1945 .

[9]  E. Rowland.  Theory of Games and Economic Behavior , 1946, Nature.

[10]  O. H. Brownlee,et al.  ACTIVITY ANALYSIS OF PRODUCTION AND ALLOCATION , 1952 .

[11]  H. Nikaidô.  On von Neumann’s minimax theorem , 1954 .

[12]  R. Freund.  THE INTRODUCTION OF RISK INTO A PROGRAMMING MODEL , 1956 .

[13]  Philip Wolfe,et al.  An algorithm for quadratic programming , 1956 .

[14]  H. Markowitz.  The optimization of a quadratic function subject to linear constraints , 1956 .

[15]  M. Marcus,et al.  Inequalities for Symmetric Functions and Hermitian Matrices , 1957, Canadian Journal of Mathematics.

[16]  Clifford Hildreth,et al.  A quadratic programming procedure , 1957 .

[17]  I. Olkin,et al.  Multivariate Chebyshev Inequalities , 1960 .

[18]  J. E. Kelley,et al.  The Cutting-Plane Method for Solving Convex Programs , 1960 .

[19]  L. V. Kantorovich,et al.  Mathematical Methods of Organizing and Planning Production , 1960 .

[20]  L. Brickman.  ON THE FIELD OF VALUES OF A MATRIX , 1961 .

[21]  K. Arrow,et al.  QUASI-CONCAVE PROGRAMMING , 1961 .

[22]  R. H. Strotz.  Theory of Value: An Axiomatic Analysis of Economic Equilibrium. , 1961 .

[23]  R. Bellman,et al.  On Systems of Linear Inequalities in Hermitian Matrix Variables , 1962 .

[24]  J. Cockcroft.  Investment in Science , 1962, Nature.

[25]  C. Davis.  Notions generalizing convexity for functions defined on spaces of matrices , 1963 .

[26]  L. Kantorovich,et al.  Functional analysis and applied mathematics , 1963 .

[27]  W. Rudin.  Principles of mathematical analysis , 1964 .

[28]  K. Isii.  Inequalities of the types of chebyshev and cramér-rao and mathematical programming , 1964 .

[29]  E. Calabi.  Linear systems of real quadratic forms. II , 1964 .

[30]  J. B. Rosen.  Pattern separation by convex programming , 1965 .

[31]  O. Mangasarian.  Linear and Nonlinear Separation of Patterns by Linear Programming , 1965 .

[32]  W. J. Studden,et al.  Tchebycheff Systems: With Applications in Analysis and Statistics. , 1967 .

[33]  Bùi-Trong-Liêu,et al.  La mèthode des centres dans un espace topologique , 1966 .

[34]  J. Ponstein,et al.  Seven kinds of convexity , 1967 .

[35]  David G. Luenberger,et al.  Quasi-Convex Programming , 1968 .

[36]  Richard F. Meyer,et al.  The Consistent Assessment and Fairing of Preference Functions , 1968, IEEE Trans. Syst. Sci. Cybern..

[37]  M. Hestenes.  Pairs of quadratic forms , 1968 .

[38]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[39]  M. J. D. Powell,et al.  Nonlinear Programming—Sequential Unconstrained Minimization Techniques , 1969 .

[40]  Adi Ben-Israel.  Linear equations and inequalities on finite dimensional, real or complex, vector spaces: A unified theory☆ , 1969 .

[41]  J. Boussard.  The introduction of risk into a programming model: different criteria and actual behaviour of farmers. , 1969 .

[42]  J. Stoer,et al.  Convexity and Optimization in Finite Dimensions I , 1970 .

[43]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[44]  A. Prékopa.  Logarithmic concave measures with applications to stochastic programming , 1971 .

[45]  Adi Ben-Israel,et al.  More on linear inequalities with applications to matrix theory , 1971 .

[46]  A. Berman.  Cones, matrices and mathematical programming , 1973 .

[47]  A. Prékopa.  On logarithmic concave measures and functions , 1973 .

[48]  R. Tyrrell Rockafellar.  Conjugate Duality and Optimization , 1974 .

[49]  D. Titterington.  Optimal design: Some geometrical aspects of D-optimality , 1975 .

[50]  Jack Elzinga,et al.  A central cutting plane algorithm for the convex programming problem , 1975, Math. Program..

[51]  P. Whittle,et al.  Optimization under Constraints , 1975 .

[52]  M. Kreĭn,et al.  The Markov Moment Problem and Extremal Problems , 1977 .

[53]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[54]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[55]  B. Craven,et al.  Generalizations of Farkas’ Theorem , 1977 .

[56]  F. Uhlig.  A recurring theorem about pairs of quadratic forms and extensions: a survey , 1979 .

[57]  P. K. Gupta,et al.  Linear programming and theory of games , 1979 .

[58]  L. G. H. Cijan.  A polynomial algorithm in linear programming , 1979 .

[59]  M. Todd,et al.  The Ellipsoid Method: A Survey , 1980 .

[60]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .

[61]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[62]  Stochastic Programming,et al.  Logarithmic Concave Measures and Related Topics , 1980 .

[63]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[64]  J. Ecker.  Geometric Programming: Methods, Computations and Applications , 1980 .

[65]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[66]  Nesa L'abbe Wu,et al.  Linear programming and extensions , 1981 .

[67]  H. Wolkowicz,et al.  Some applications of optimization in matrix theory , 1981 .

[68]  Philip E. Gill,et al.  Practical optimization , 1981 .

[69]  Harold W. Kuhn,et al.  Nonlinear programming: a historical view , 1982, SMAP.

[70]  J. Pasciak,et al.  Computer solution of large sparse positive definite systems , 1982 .

[71]  Siegfried Schaible,et al.  Bibliography in fractional programming , 1982, Z. Oper. Research.

[72]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[73]  W. Fenchel.  Convexity Through the Ages , 1983 .

[74]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[75]  Jan van Tiel.  Convex Analysis: An Introductory Text , 1984 .

[76]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, Comb..

[77]  J. Gower.  Properties of Euclidean and non-Euclidean distance matrices , 1985 .

[78]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[79]  Michael A. Saunders,et al.  On projected newton barrier methods for linear programming and an equivalence to Karmarkar’s projective method , 1986, Math. Program..

[80]  G. Sonnevend.  An "analytical centre" for polyhedrons and new classes of global algorithms for linear (smooth, convex) programming , 1986 .

[81]  Aharon Ben-Tal,et al.  Lectures on modern convex optimization , 1987 .

[82]  Evanghelos Zafiriou,et al.  Robust process control , 1987 .

[83]  I. Duff,et al.  Direct Methods for Sparse Matrices , 1987 .

[84]  G. Stewart,et al.  Theory of the Combination of Observations Least Subject to Errors , 1987 .

[85]  A. Peressini,et al.  The Mathematics Of Nonlinear Programming , 1988 .

[86]  David Kendrick,et al.  GAMS, a user's guide , 1988, SGNM.

[87]  J. G. Pierce,et al.  Geometric Algorithms and Combinatorial Optimization , 2016 .

[88]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[89]  Konrad Doll,et al.  Analytical placement: a linear or a quadratic objective function? , 1991, 28th ACM/IEEE Design Automation Conference.

[90]  Stephen P. Boyd,et al.  Linear controller design: limits of performance , 1991 .

[91]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[92]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[93]  S. Vavasis.  Nonlinear optimization: complexity issues , 1991 .

[94]  Georg Sigl,et al.  GORDIAN: VLSI placement by quadratic programming and slicing optimization , 1991, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[95]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[96]  Clóvis C. Gonzaga,et al.  Path-Following Methods for Linear Programming , 1992, SIAM Rev..

[97]  Sanjay Mehrotra,et al.  On the Implementation of a Primal-Dual Interior Point Method , 1992, SIAM J. Optim..

[98]  Audra E. Kosh,et al.  Linear Algebra and its Applications , 1992 .

[99]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[100]  Iain S. Duff,et al.  The solution of augmented systems , 1993 .

[101]  Brian W. Kernighan,et al.  AMPL: A Modeling Language for Mathematical Programming , 1993 .

[102]  Sung-Mo Kang,et al.  An exact solution to the transistor sizing problem for CMOS circuits using convex optimization , 1993, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[103]  R. Tyrrell Rockafellar,et al.  Lagrange Multipliers and Optimality , 1993, SIAM Rev..

[104]  Marvin Marcus,et al.  A Convex Set , 1993, SIAM Rev..

[105]  Stephen P. Boyd,et al.  Method of centers for minimizing generalized eigenvalues , 1993, Linear Algebra and its Applications.

[106]  Robert J. Plemmons,et al.  Nonnegative Matrices in the Mathematical Sciences , 1979, Classics in Applied Mathematics.

[107]  Michael Jackson,et al.  Optimal Design of Experiments , 1994 .

[108]  Roy E. Marsten,et al.  Feature Article - Interior Point Methods for Linear Programming: Computational State of the Art , 1994, INFORMS J. Comput..

[109]  F. Jarre.  Optimal ellipsoidal approximations around the Analytic center , 1994 .

[110]  Dick den Hertog,et al.  Interior Point Approach to Linear, Quadratic and Convex Programming: Algorithms and Complexity , 1994 .

[111]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[112]  Shinji Mizuno,et al.  An O(√nL)-Iteration Homogeneous and Self-Dual Linear Programming Algorithm , 1994, Math. Oper. Res..

[113]  P. Pardalos,et al.  Handbook of global optimization , 1995 .

[114]  H. Fawcett.  Manual of Political Economy , 1995 .

[115]  M. Dahleh,et al.  Control of Uncertain Systems: A Linear Programming Approach , 1995 .

[116]  János D. Pintér,et al.  Global optimization in action , 1995 .

[117]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[118]  J. Lasserre.  A new Farkas lemma for positive semidefinite matrices , 1995, IEEE Trans. Autom. Control..

[119]  Henry Wolkowicz,et al.  Indefinite Trust Region Subproblems and Nonsymmetric Eigenvalue Perturbations , 1995, SIAM J. Optim..

[120]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[121]  A. Mas-Colell,et al.  Microeconomic Theory , 1995 .

[122]  Yinyu Ye,et al.  Complexity Analysis of an Interior Cutting Plane Method for Convex Feasibility Problems , 1996, SIAM J. Optim..

[123]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[124]  Yinyu Ye,et al.  A simplified homogeneous and self-dual linear programming algorithm and its implementation , 1996, Ann. Oper. Res..

[125]  Robert J. Vanderbei,et al.  An Interior-Point Method for Semidefinite Programming , 1996, SIAM J. Optim..

[126]  Per Christian Hansen,et al.  Rank-Deficient and Discrete Ill-Posed Problems , 1996 .

[127]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[128]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[129]  Stephen J. Wright,et al.  Superlinear convergence of an interior-point method for monotone variational inequalities , 1996 .

[130]  L. Faybusovich.  Linear systems in Jordan algebras and primal-dual interior-point algorithms , 1997 .

[131]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[132]  Stephen J. Wright.  Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[133]  Renato D. C. Monteiro,et al.  Primal-Dual Path-Following Algorithms for Semidefinite Programming , 1997, SIAM J. Optim..

[134]  L. Faybusovich.  Euclidean Jordan Algebras and Interior-point Algorithms , 1997 .

[135]  John N. Tsitsiklis,et al.  Introduction to linear optimization , 1997, Athena scientific optimization and computation series.

[136]  Xiaoye S. Li,et al.  SuperLU Users'' Guide , 1997 .

[137]  C. Scherer,et al.  Multiobjective output-feedback control via LMI optimization , 1997, IEEE Trans. Autom. Control..

[138]  Laurent El Ghaoui,et al.  Robust Solutions to Least-Squares Problems with Uncertain Data , 1997, SIAM J. Matrix Anal. Appl..

[139]  Shinji Hara,et al.  Interior-Point Methods for the Monotone Semidefinite Linear Complementarity Problem in Symmetric Matrices , 1997, SIAM J. Optim..

[140]  Yinyu Ye,et al.  A Computational Study of the Homogeneous Algorithm for Large-scale Convex Optimization , 1998, Comput. Optim. Appl..

[141]  Jean-Philippe Vial,et al.  Theory and algorithms for linear optimization - an interior point approach , 1998, Wiley-Interscience series in discrete mathematics and optimization.

[142]  Y. Nesterov.  Semidefinite relaxation and nonconvex quadratic optimization , 1998 .

[143]  Laurent El Ghaoui,et al.  Robust Solutions to Uncertain Semidefinite Programs , 1998, SIAM J. Optim..

[144]  Robert J. Vanderbei,et al.  Linear Programming: Foundations and Extensions , 1998, Kluwer international series in operations research and management service.

[145]  Arkadi Nemirovski,et al.  Robust Convex Optimization , 1998, Math. Oper. Res..

[146]  Stephen J. Wright,et al.  Application of Interior-Point Methods to Model Predictive Control , 1998 .

[147]  E. Yaz.  Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.

[148]  Yinyu Ye,et al.  Interior point algorithms: theory and analysis , 1997 .

[149]  Yin Zhang,et al.  On Extending Some Primal-Dual Interior-Point Algorithms From Linear Programming to Semidefinite Programming , 1998, SIAM J. Optim..

[150]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[151]  Z. Luo,et al.  Conic convex programming and self-dual embedding , 1998 .

[152]  Sabih H. Gerez,et al.  Algorithms for VLSI design automation , 1998 .

[153]  G. Golub,et al.  Parameter Estimation in the Presence of Bounded Data Uncertainties , 1998 .

[154]  Kim-Chuan Toh,et al.  On the Nesterov-Todd Direction in Semidefinite Programming , 1998, SIAM J. Optim..

[155]  S. Ross.  An Introduction to Mathematical Finance: Options and Other Topics , 1999 .

[156]  Kumaraswamy Ponnambalam,et al.  A unified approach to statistical design centering of integrated circuits with correlated parameters , 1999 .

[157]  Garret N. Vanderplaats,et al.  Numerical optimization techniques for engineering design , 1999 .

[158]  Arkadi Nemirovski,et al.  Robust solutions of uncertain linear programs , 1999, Oper. Res. Lett..

[159]  Alexander Schrijver,et al.  Theory of linear and integer programming , 1986, Wiley-Interscience series in discrete mathematics and optimization.

[160]  Jason Wu,et al.  The Reference Manual for SPOOLES, Release 2.2: An Object Oriented Software Library for Solving Sparse Linear Systems of Equations , 1999 .

[161]  Yinyu Ye,et al.  Approximating quadratic programming with bound and quadratic constraints , 1999, Math. Program..

[162]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[163]  Laurent El Ghaoui,et al.  Advances in linear matrix inequality methods in control: advances in design and control , 1999 .

[164]  Er-Wei Bai,et al.  Bounded error parameter estimation: a sequential analytic center approach , 1999, IEEE Trans. Autom. Control..

[165]  Yurii Nesterov,et al.  Squared Functional Systems and Optimization Problems , 2000 .

[166]  Zhi-Quan Luo,et al.  Design of orthogonal pulse shapes for communications via semidefinite programming , 2000, IEEE Trans. Signal Process..

[167]  Adrian S. Lewis,et al.  Convex Analysis And Nonlinear Optimization , 2000 .

[168]  Henry Wolkowicz,et al.  Handbook of Semidefinite Programming , 2000 .

[169]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[170]  Russ Bubley,et al.  Randomized algorithms , 1995, CSUR.

[171]  P. Parrilo.  Structured semidefinite programs and semialgebraic geometry methods in robustness and optimization , 2000 .

[172]  Y. Ye,et al.  Semidefinite programming relaxations of nonconvex quadratic optimization , 2000 .

[173]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[174]  D. Bertsimas,et al.  Moment Problems and Semidefinite Optimization , 2000 .

[175]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[176]  M. Florenzano,et al.  Finite Dimensional Convexity and Optimization , 2001 .

[177]  Stephen P. Boyd,et al.  Optimal design of a CMOS op-amp via geometric programming , 2001, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[178]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[179]  Leonid Faybusovich,et al.  A long-step primal-dual algorithm for the symmetric programming problem , 2001, Syst. Control. Lett..

[180]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[181]  James Renegar,et al.  A mathematical view of interior-point methods in convex optimization , 2001, MPS-SIAM series on optimization.

[182]  Michael J. Todd,et al.  The many facets of linear programming , 2002, Math. Program..

[183]  Alexander Barvinok,et al.  A course in convexity , 2002, Graduate studies in mathematics.

[184]  Steven J. Benson,et al.  DSDP4 { A Software Package Implementing the Dual-Scaling Algorithm for Semidenite Programming 1 , 2002 .

[185]  Anders Forsgren,et al.  Interior Methods for Nonlinear Optimization , 2002, SIAM Rev..

[186]  Jiming Peng,et al.  Self-regularity - a new paradigm for primal-dual interior-point algorithms , 2002, Princeton series in applied mathematics.

[187]  J. Lasserre.  Bounds on measures satisfying moment conditions , 2002 .

[188]  Zhi-Quan Luo,et al.  Quasi-maximum-likelihood multiuser detection using semi-definite relaxation with application to synchronous CDMA , 2002, IEEE Trans. Signal Process..

[189]  Takashi Tsuchiya,et al.  Primal-dual algorithms and infinite-dimensional Jordan algebras of finite rank , 2003, Math. Program..

[190]  Pablo A. Parrilo,et al.  Semidefinite programming relaxations for semialgebraic problems , 2003, Math. Program..

[191]  Donald Goldfarb,et al.  Robust Portfolio Selection Problems , 2003, Math. Oper. Res..

[192]  Donald Goldfarb,et al.  Robust convex quadratically constrained programs , 2003, Math. Program..

[193]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[194]  Donald Goldfarb,et al.  Second-order cone programming , 2003, Math. Program..

[195]  John P. Fishburn,et al.  TILOS: A posynomial programming approach to transistor sizing , 2003, ICCAD 2003.

[196]  Zhi-Quan Luo,et al.  Applications of convex optimization in signal processing and digital communication , 2003, Math. Program..

[197]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[198]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[199]  J. Lofberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004, 2004 IEEE International Conference on Robotics and Automation (IEEE Cat. No.04CH37508).

[200]  Stephen P. Boyd,et al.  Fastest Mixing Markov Chain on a Graph , 2004, SIAM Rev..

[201]  M. H. Wright.  The interior-point revolution in optimization: History, recent developments, and lasting consequences , 2004 .

[202]  G. Dullerud,et al.  A Course in Robust Control Theory: A Convex Approach , 2005 .

[203]  A. Banerjee.  Convex Analysis and Optimization , 2006 .

[204]  R. Kannan,et al.  Convex Sets and their Applications , 2006 .

[205]  E. Beckenbach.  CONVEX FUNCTIONS , 2007 .

[206]  Katta G. Murty,et al.  Nonlinear Programming Theory and Algorithms , 2007, Technometrics.

[207]  I. Bárány,et al.  convex sets , 2007 .

[208]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .