A Fresh Variational-Analysis Look at the Positive Semidefinite Matrices World

Engineering sciences and applications of mathematics show unambiguously that positive semidefiniteness of matrices is the most important generalization of non-negative real numbers. This notion of non-negativity for matrices has been well-studied in the literature; it has been the subject of review papers and entire chapters of books.This paper reviews some of the nice, useful properties of positive (semi)definite matrices, and insists in particular on (i) characterizations of positive (semi)definiteness and (ii) the geometrical properties of the set of positive semidefinite matrices. Some properties that turn out to be less well-known have here a special treatment. The use of these properties in optimization, as well as various references to applications, is spread all the way through.The “raison d’être” of this paper is essentially pedagogical; it adopts the viewpoint of variational analysis, shedding new light on the topic. Important, fruitful, and subtle, the positive semidefinite world is a good place to start with this domain of applied mathematics.

[1]  Eugene P. Wigner,et al.  On Weakly Positive Matrices , 1963, Canadian Journal of Mathematics.

[2]  A. Lewis.  Eigenvalue-constrained faces☆ , 1998 .

[3]  Adrian S. Lewis,et al.  Convex Analysis And Nonlinear Optimization , 2000 .

[4]  Jan Brinkhuis,et al.  Optimization: Insights and Applications (Princeton Series in Applied Mathematics) , 2005 .

[5]  Franz Rendl,et al.  A recipe for semidefinite relaxation for (0,1)-quadratic programming , 1995, J. Glob. Optim..

[6]  Clifford Ambrose Truesdell,et al.  A first course in rational continuum mechanics , 1976 .

[7]  Nathan Krislock,et al.  Explicit Sensor Network Localization using Semidefinite Representations and Facial Reductions , 2010, SIAM J. Optim..

[8]  J. Hiriart-Urruty,et al.  Fundamentals of Convex Analysis , 2004 .

[9]  J. Lasserre,et al.  Handbook on Semidefinite, Conic and Polynomial Optimization , 2012 .

[10]  R. Duffin,et al.  Series and parallel addition of matrices , 1969 .

[11]  Maher Moakher,et al.  Symmetric Positive-Definite Matrices: From Geometry to Applications and Visualization , 2006, Visualization and Processing of Tensor Fields.

[12]  Cornelis Vuik,et al.  On deflation and singular symmetric positive semi-definite matrices , 2007 .

[13]  J. Borwein,et al.  Facial reduction for a cone-convex programming problem , 1981, Journal of the Australian Mathematical Society. Series A. Pure Mathematics and Statistics.

[14]  Hans Zassenhaus,et al.  On the similarity transformation between a matirx and its transpose. , 1959 .

[15]  Adrian S. Lewis,et al.  Nonsmooth analysis of eigenvalues , 1999, Math. Program..

[16]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[17]  Michael L. Overton,et al.  Optimality conditions and duality theory for minimizing sums of the largest eigenvalues of symmetric matrices , 2015, Math. Program..

[18]  Jérôme Malick,et al.  A Dual Approach to Semidefinite Least-Squares Problems , 2004, SIAM J. Matrix Anal. Appl..

[19]  Olga Taussky,et al.  The role of symmetric matrices in the study of general matrices , 1972 .

[20]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[21]  C. Ballantine,et al.  Products of positive definite matrices. III , 1968 .

[22]  J. Hiriart-Urruty,et al.  Sensitivity analysis of all eigenvalues of a symmetric matrix , 1995 .

[23]  Michael J. Todd,et al.  On the Riemannian Geometry Defined by Self-Concordant Barriers and Interior-Point Methods , 2002, Found. Comput. Math..

[24]  Sylvie Benzoni-Gavage.  Calcul différentiel et équations différentielles : Cours et exercices corrigés Ed. 2 , 2014 .

[25]  Vladimir A. Yakubovich,et al.  Linear Matrix Inequalities in System and Control Theory (S. Boyd, L. E. Ghaoui, E. Feron, and V. Balakrishnan) , 1995, SIAM Rev..

[26]  Xavier Pennec,et al.  Statistical Computing on Manifolds for Computational Anatomy , 2006 .

[27]  J. Crouzeix,et al.  Definiteness and semidefiniteness of quadratic forms revisited , 1984 .

[28]  Maher Moakher,et al.  A Differential Geometric Approach to the Geometric Mean of Symmetric Positive-Definite Matrices , 2005, SIAM J. Matrix Anal. Appl..

[29]  Jean-Baptiste Hiriart-Urruty,et al.  Potpourri of Conjectures and Open Questions in Nonlinear Analysis and Optimization , 2007, SIAM Rev..

[30]  Jean Charles Gilbert,et al.  Some Theory of Nonsmooth Optimization , 2003 .

[31]  Jan Brinkhuis,et al.  Optimization Insights and Applications , 2010 .

[32]  Yurii Nesterov,et al.  Smoothing Technique and its Applications in Semidefinite Optimization , 2004, Math. Program..

[33]  Florent Cadoux.  Optimisation et analyse convexe pour la dynamique non-régulière , 2009 .

[34]  Michael Merritt,et al.  Plenary Talk , 2004, Proceedings of the IEEE International Conference on Mechatronics, 2004. ICM '04..

[35]  Silvere Bonnabel,et al.  Riemannian Metric and Geometric Mean for Positive Semidefinite Matrices of Fixed Rank , 2008, SIAM J. Matrix Anal. Appl..

[36]  Marie-Françoise Roy,et al.  Real algebraic geometry , 1992 .

[37]  Ernst Kunz.  Erich Kähler. mathematische werke. mathematical works , 2006 .

[38]  Adrian S. Lewis,et al.  Alternating Projections on Manifolds , 2008, Math. Oper. Res..

[39]  David W. Lewis,et al.  Matrix theory , 1991 .

[40]  Alberto Seeger,et al.  A Variational Approach to Copositive Matrices , 2010, SIAM Rev..

[41]  Henry Wolkowicz,et al.  Handbook of Semidefinite Programming , 2000 .

[42]  Heinz H. Bauschke,et al.  The resolvent average for positive semidefinite matrices , 2009, 0910.3705.

[43]  Xinmin Wang.  Comparison theorems for a class of parallel multisplitting AOR type iterative methods , 1998 .

[44]  Jan Brinkhuis,et al.  Optimization: Insights and Applications: Insights and Applications , 2005 .

[45]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[46]  Ivar Ekeland.  Some applications of the Cartan-Kähler theorem to economic theory , 2005 .

[47]  Jean-Baptiste Hiriart-Urruty,et al.  A new series of conjectures and open questions in optimization and matrix analysis , 2009 .

[48]  O. Mangasarian,et al.  Conjugate Cone Characterization of Positive Definite and Semidefinite Matrices. , 1984 .

[49]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[50]  Adrian S. Lewis,et al.  The [barred L]ojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems , 2006, SIAM J. Optim..

[51]  G. Strang.  Introduction to Linear Algebra , 1993 .

[52]  A. D. Ioffe,et al.  An Invitation to Tame Optimization , 2008, SIAM J. Optim..

[53]  Hans Hagen,et al.  Visualization and Processing of Tensor Fields , 2014 .

[54]  E. Yaz.  Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.

[55]  Michael I. Jordan,et al.  A Direct Formulation for Sparse Pca Using Semidefinite Programming , 2004, SIAM Rev..

[56]  Neil C. Schwertman,et al.  Smoothing an indefinite variance-covariance matrix , 1979 .

[57]  A. J. Bosch,et al.  The factorization of a square matrix into two symmetric matrices , 1986 .

[58]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[59]  R. Saigal,et al.  Handbook of semidefinite programming : theory, algorithms, and applications , 2000 .

[60]  R. Bhatia.  Positive Definite Matrices , 2007 .

[61]  Stephen P. Boyd,et al.  Linear Matrix Inequalities in Systems and Control Theory , 1994 .