Determinant Maximization with Linear Matrix Inequality Constraints

The problem of maximizing the determinant of a matrix subject to linear matrix inequalities (LMIs) arises in many fields, including computational geometry, statistics, system identification, experiment design, and information and communication theory. It can also be considered as a generalization of the semidefinite programming problem. We give an overview of the applications of the determinant maximization problem, pointing out simple cases where specialized algorithms or analytical solutions are known. We then describe an interior-point method, with a simplified analysis of the worst-case complexity and numerical results that indicate that the method is very efficient, both in theory and in practice. Compared to existing specialized algorithms (where they are available), the interior-point method will generally be slower; the advantage is that it handles a much wider variety of problems.

[1]  V. Klee,et al.  Helly's theorem and its relatives , 1963 .

[2]  J. B. Rosen Pattern separation by convex programming , 1965 .

[3]  F. Schweppe Recursive state estimation: Unknown but bounded errors and system inputs , 1967 .

[4]  W. J. Studden,et al.  Tchebycheff Systems: With Applications in Analysis and Statistics. , 1967 .

[5]  H. Witsenhausen Sets of possible states of linear systems given perturbed observations , 1968 .

[6]  D. Luenberger Optimization by Vector Space Methods , 1968 .

[7]  M. J. D. Powell,et al.  Nonlinear Programming—Sequential Unconstrained Minimization Techniques , 1969 .

[8]  D. Bertsekas,et al.  Recursive state estimation for a set-membership description of uncertainty , 1971 .

[9]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[10]  G. Golub,et al.  Bounds for the error of linear systems of equations using the theory of moments , 1972 .

[11]  Fred C. Schweppe,et al.  Uncertain dynamic systems , 1973 .

[12]  D. Titterington Optimal design: Some geometrical aspects of D-optimality , 1975 .

[13]  N. Z. Shor Cut-off method with space extension in convex programming problems , 1977, Cybernetics.

[14]  M. Kreĭn,et al.  The Markov Moment Problem and Extremal Problems , 1977 .

[15]  丸山 徹 Convex Analysisの二,三の進展について , 1977 .

[16]  E. Fogel System identification via membership set constraints with energy constrained noise , 1979 .

[17]  H. Dym,et al.  Extensions of band matrices with band inverses , 1981 .

[18]  Peter Whittle,et al.  Optimization Over Time , 1982 .

[19]  D. Luenberger,et al.  Estimation of structured covariance matrices , 1982, Proceedings of the IEEE.

[20]  Y. F. Huang,et al.  On the value of information in system identification - Bounded noise case , 1982, Autom..

[21]  E. Barnes An algorithm for separating patterns by ellipsoids , 1982 .

[22]  W. Kahan,et al.  NORM-PRESERVING DILATIONS AND THEIR APPLICATIONS TO OPTIMAL ERROR BOUNDS* , 1982 .

[23]  R. Hartley,et al.  Optimisation Over Time: Dynamic Programming and Stochastic Control: , 1983 .

[24]  G. Grammin Polynomial-time Algorithm , 1984 .

[25]  Charles R. Johnson,et al.  Positive definite completions of partial Hermitian matrices , 1984 .

[26]  Edward J. Wegman,et al.  Statistical Signal Processing , 1985 .

[27]  G. Sonnevend An "analytical centre" for polyhedrons and new classes of global algorithms for linear (smooth, convex) programming , 1986 .

[28]  Amir Dembo,et al.  The relation between maximum likelihood estimation of structured covariance matrices and periodograms , 1986, IEEE Trans. Acoust. Speech Signal Process..

[29]  J. P. Norton,et al.  Identification and application of bounded-parameter models , 1985, Autom..

[30]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[31]  I. Gohberg Topics in Operator Theory and Interpolation , 1988 .

[32]  P Dewilde,et al.  The Generalized Schur Algorithm: Approximation and Hierarchy , 1988 .

[33]  James Renegar,et al.  A polynomial-time algorithm, based on Newton's method, for linear programming , 1988, Math. Program..

[34]  Charles R. Johnson,et al.  Determinantal formulae for matrix completions associated with chordal graphs , 1989 .

[35]  J. Deller Set membership identification in digital signal processing , 1989, IEEE ASSP Magazine.

[36]  J. Nocedal,et al.  A tool for the analysis of Quasi-Newton methods with application to unconstrained minimization , 1989 .

[37]  Colin L. Mallows,et al.  Embedding nonnegative definite Toeplitz matrices in nonnegative definite circulant matrices, with application to covariance estimation , 1989, IEEE Trans. Inf. Theory.

[38]  Thomas M. Cover,et al.  Gaussian feedback capacity , 1989, IEEE Trans. Inf. Theory.

[39]  Thomas Kailath,et al.  Signal processing Part I: signal processing theory , 1990 .

[40]  E. Walter,et al.  Estimation of parameter bounds from bounded-error data: a survey , 1990 .

[41]  Charles R. Johnson Positive definite completions: a guide to selected literature , 1990 .

[42]  Patrick Dewilde,et al.  Models for Large Integrated Circuits , 1990 .

[43]  Roger Fletcher,et al.  A New Variational Result for Quasi-Newton Formulae , 1991, SIAM J. Optim..

[44]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[45]  György Sonnevend Applications of Analytic Centers , 1991 .

[46]  Charles R. Johnson,et al.  Linearly constrained positive definite completions , 1991 .

[47]  John M. Cioffi,et al.  Achievable information rates on digital subscriber loops: limiting information rates with crosstalk noise , 1992, IEEE Trans. Commun..

[48]  Geir Nævdal,et al.  Partial matrix contractions and intersections of matrix balls , 1992 .

[49]  Sachin Suresh Sapatnekar A convex programming approach to problems in VLSI design , 1992 .

[50]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[51]  F. Chernousko State Estimation for Dynamic Systems , 1993 .

[52]  Leonid Khachiyan,et al.  On the complexity of approximating the maximal inscribed ellipsoid for a polytope , 1993, Math. Program..

[53]  J. Dennis,et al.  Sizing and least-change secant methods , 1993 .

[54]  J. Helton,et al.  Symmetric Hankel operators: minimal norm extensions and eigenstructures , 1993 .

[55]  J. R. Deller,et al.  Least-square identification with error bounds for real-time signal processing and control , 1993, Proc. IEEE.

[56]  Stephen P. Boyd,et al.  Method of centers for minimizing generalized eigenvalues , 1993, Linear Algebra and its Applications.

[57]  K. Passino,et al.  An optimal volume ellipsoid algorithm for parameter set estimation , 1993, IEEE Trans. Autom. Control..

[58]  Michael Jackson,et al.  Optimal Design of Experiments , 1994 .

[59]  Y. Nesterov,et al.  Self-Scaled Cones and Interior-Point Methods in Nonlinear Programming , 1994 .

[60]  Dick den Hertog,et al.  Interior Point Approach to Linear, Quadratic and Convex Programming: Algorithms and Complexity , 1994 .

[61]  Eric Walter,et al.  Minimum-volume ellipsoids containing compact sets : Application to parameter bounding , 1994, Autom..

[62]  Stephen P. Boyd,et al.  A primal—dual potential reduction method for problems involving matrix inequalities , 1995, Math. Program..

[63]  Dennis Cook,et al.  Constrained Optimization of Experimental Design , 1995 .

[64]  Alexander Shapiro,et al.  On Eigenvalue Optimization , 1995, SIAM J. Optim..

[65]  Hugo J. Woerdeman,et al.  Maximum Entropy Elements in the Intersection of an Affine Space and the Cone of Positive Definite Matrices , 1995, SIAM J. Matrix Anal. Appl..

[66]  Farid Alizadeh,et al.  Interior Point Methods in Semidefinite Programming with Applications to Combinatorial Optimization , 1995, SIAM J. Optim..

[67]  Stephen Boyd,et al.  MAXDET: Software for Determinant Maximization Problems User's Guide , 1996 .

[68]  Adrian S. Lewis,et al.  Convex Analysis on the Hermitian Matrices , 1996, SIAM J. Optim..

[69]  Elizabeth Schwerer A linear programming approach to the steady-state analysis of Markov processes , 1996 .

[71]  Jon Lee,et al.  Continuous Relaxations for Constrained Maximum-Entropy Sampling , 1996, IPCO.

[72]  A. Wilhelm COMPUTING OPTIMAL DESIGNS BY BUNDLE TRUST METHODS , 1997 .

[73]  Jon Lee Constrained Maximum-Entropy Sampling , 1998, Oper. Res..

[74]  Henry Wolkowicz,et al.  An Interior-Point Method for Approximate Positive Semidefinite Completions , 1998, Comput. Optim. Appl..

[75]  E. Yaz Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.

[76]  Stephen P. Boyd,et al.  Connections between Semi-Infinite and Semidefinite Programming , 1998 .