A Majorized Penalty Approach for Calibrating Rank Constrained Correlation Matrix Problems

In this paper, we aim at finding a nearest correlation matrix to a given symmetric matrix, measured by the componentwise weighted Frobenius norm, with a prescribed rank and bound constraints on its correlations. This is in general a non-convex and difficult problem due to the presence of the rank constraint. To deal with this difficulty, we first consider a penalized version of this problem and then apply the essential ideas of the majorization method to the penalized problem by solving iteratively a sequence of least squares correlation matrix problems without the rank constraint. The latter problems can be solved by a recently developed quadratically convergent smoothing Newton-BiCGStab method. Numerical examples demonstrate that our approach is very efficient for obtaining a nearest correlation matrix with both rank and bound constraints.

[1]  K. Fan.  On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations I. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[2]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[3]  Neil C. Schwertman,et al.  Smoothing an indefinite variance-covariance matrix , 1979 .

[4]  George Cybenko,et al.  Moment problems and low rank Toeplitz approximations , 1982 .

[5]  D. Luenberger,et al.  Estimation of structured covariance matrices , 1982, Proceedings of the IEEE.

[6]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[7]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[8]  B. Flury.  Common Principal Components and Related Multivariate Models , 1988 .

[9]  J. Leeuw.  Convergence of the majorization method for multidimensional scaling , 1988 .

[10]  Henk A. L. Kiers,et al.  Majorization as a tool for optimizing a class of matrix functions , 1990 .

[11]  W. Heiser.  A generalized majorization method for least souares multidimensional scaling of pseudodistances that may be negative , 1991 .

[12]  Michael L. Overton,et al.  Optimality conditions and duality theory for minimizing sums of the largest eigenvalues of symmetric matrices , 2015, Math. Program..

[13]  Jan de Leeuw,et al.  Block-relaxation Algorithms in Statistics , 1994 .

[14]  D. A. Wolf.  Recent advances in descriptive multivariate analysis , 1996 .

[15]  A. S. Lewis,et al.  Derivatives of Spectral Functions , 1996, Math. Oper. Res..

[16]  P. Groenen,et al.  Modern multidimensional scaling , 1996 .

[17]  Philip M. Lurie,et al.  An Approximate Method for Sampling Correlated Random Variables From Partially-Specified Distributions , 1998 .

[18]  Riccardo Rebonato,et al.  On the simultaneous calibration of multifactor lognormal interest rate models to Black volatilities and to the correlation matrix , 1999 .

[19]  R. Rebonato,et al.  The Most General Methodology to Create a Valid Correlation Matrix for Risk Management and Option Pricing Purposes , 2011 .

[20]  J. Leeuw.  Applications of Convex Analysis to Multidimensional Scaling , 2000 .

[21]  D. Brigo,et al.  Interest Rate Models , 2001 .

[22]  N. Higham.  Computing the nearest correlation matrix—a problem from finance , 2002 .

[23]  D. Brigo,et al.  A Note on Correlation and Rank Reduction , 2002 .

[24]  Henk A. L. Kiers,et al.  Setting up alternating least squares and iterative majorization algorithms for solving various matrix optimization problems , 2002, Comput. Stat. Data Anal..

[25]  Miriam Hodge.  Fast at-the-money calibration of the Libor market model using Lagrange multipliers , 2003 .

[26]  T. Allen.  Thank you. , 2003, CJEM.

[27]  Zhenyue Zhang,et al.  Optimal low-rank approximation to a correlation matrix , 2003 .

[28]  R. Plemmons,et al.  Structured low rank approximation , 2003 .

[29]  Alexandre d'Aspremont.  Interest rate model calibration using semidefinite Programming , 2003, ArXiv.

[30]  William Scott Hoge,et al.  A subspace identification extension to the phase correlation method [MRI application] , 2003, IEEE Transactions on Medical Imaging.

[31]  Sudhanshu K. Mishra.  Optimal Solution of the Nearest Correlation Matrix Problem by Minimization of the Maximum Norm , 2004 .

[32]  Nick Webber,et al.  An EZI Method to Reduce the Rank of a Correlation Matrix in Financial Modelling , 2004 .

[33]  Igor Grubisic,et al.  Efficient Rank Reduction of Correlation Matrices , 2004, cond-mat/0403477.

[34]  P. Groenen,et al.  Rank reduction of correlation matrices by majorization , 2004 .

[35]  G. Alistair Watson,et al.  On matrix approximation problems with Ky Fank norms , 1993, Numerical Algorithms.

[36]  Stephen P. Boyd,et al.  Least-Squares Covariance Matrix Adjustment , 2005, SIAM J. Matrix Anal. Appl..

[37]  R N Mantegna,et al.  Spectral density of the correlation matrix of factor models: a random matrix theory approach. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[38]  Jan de Leeuw.  A Decomposition Method for Weighted Least Squares Low-rank Approximation of Symmetric Matrices , 2006 .

[39]  Defeng Sun,et al.  A Quadratically Convergent Newton Method for Computing the Nearest Correlation Matrix , 2006, SIAM J. Matrix Anal. Appl..

[40]  Dan Simon,et al.  Reduced Order Kalman Filtering without Model Reduction , 2007, Control. Intell. Syst..

[41]  D. Brigo,et al.  Parameterizing correlations: a geometric interpretation , 2007 .

[42]  Ralf Werner,et al.  Calibration of correlation matrices - SDP or not SDP , 2007 .

[43]  R. Jackson.  Inequalities , 2007, Algebra for Parents.

[44]  M. Overton.  NONSMOOTH OPTIMIZATION VIA BFGS , 2008 .

[45]  Alec N. Kercheval.  ON REBONATO AND JÄCKEL ’ S PARAMETRIZATION METHOD FOR FINDING NEAREST CORRELATION MATRICES , 2008 .

[46]  M. Fielden,et al.  The liver pharmacological and xenobiotic gene response repertoire , 2008, Molecular systems biology.

[47]  Ali Burak Kurtulan,et al.  Correlations in Economic Capital Models for Pension Fund Pooling , 2009 .

[48]  Yan Gao,et al.  Calibrating Least Squares Semidefinite Programming with Equality and Inequality Constraints , 2009, SIAM J. Matrix Anal. Appl..

[49]  Tadayoshi Fushiki,et al.  Estimation of Positive Semidefinite Correlation Matrices by Using Convex Quadratic Semidefinite Programming , 2009, Neural Computation.

[50]  D. Simon,et al.  Author's Personal Copy Linear Algebra and Its Applications a Majorization Algorithm for Constrained Correlation Matrix Approximation , 2022 .

[51]  Defeng Sun,et al.  Correlation stress testing for value-at-risk: an unconstrained convex optimization approach , 2010, Comput. Optim. Appl..

[52]  Y. D. Chen,et al.  An Inexact SQP Newton Method for Convex SC1 Minimization Problems , 2010 .

[53]  Houduo Qi,et al.  A Sequential Semismooth Newton Method for the Nearest Low-rank Correlation Matrix Problem , 2011, SIAM J. Optim..

[54]  H. Qi,et al.  An augmented Lagrangian dual approach for the H-weighted nearest correlation matrix problem , 2011 .