Covariance Versus Precision Matrix Estimation for Efficient Asset Allocation

Asset allocation constitutes one of the most crucial and most challenging tasks in financial engineering, which often requires the estimation of large covariance or precision matrices, from short time-span multivariate observations, a mandatory yet difficult step. The present contribution reviews and compares a large selection of estimators for covariance and precision matrices, organized into classes of estimation principles (direct, factor, shrinkage, sparsity). This includes the theoretical derivation of several additional estimators not available in the literature. Rather than assessing estimation performance from synthetic data based on a priori selected models of questionable practical interest, it is chosen here to evaluate practically the quality of these estimators directly from portfolio selection performance, quantified by financial criteria. Portfolio selection is conducted over two datasets of different natures: a 15-year large subset (within Stoxx Europe 600) of 244 European stock returns, and a 50-year benchmark dataset of 90 US equity portfolios. This large scale comparative study addresses issues such as the relative benefits and difficulties of using robust versus direct estimates, of choosing precision or covariance estimates, of quantifying the impacts of constraints.

[1]  E. Elton,et al.  ESTIMATING THE DEPENDENCE STRUCTURE OF SHARE PRICES —IMPLICATIONS FOR PORTFOLIO SELECTION , 1973 .

[2]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[3]  A. Hero,et al.  Large-Scale Correlation Screening , 2011, 1102.1204.

[4]  Ali N. Akansu,et al.  A Primer for Financial Engineering: Financial Signal Processing and Electronic Trading , 2015 .

[5]  Noureddine El Karoui.  Spectrum estimation for large dimensional covariance matrices using random matrix theory , 2006, math/0609418.

[6]  Taras Bodnar,et al.  Properties of the singular, inverse and generalized inverse partitioned Wishart distributions , 2008 .

[7]  W. Sharpe.  A Simplified Model for Portfolio Analysis , 1963 .

[8]  Alfred O. Hero,et al.  Robust Shrinkage Estimation of High-Dimensional Covariance Matrices , 2010, IEEE Transactions on Signal Processing.

[9]  Tatsuya Kubokawa,et al.  Estimation of the precision matrix of a singular Wishart distribution and its application in high-dimensional data , 2008 .

[10]  Pradeep Ravikumar,et al.  Elementary Estimators for Sparse Covariance Matrices and other Structured Moments , 2014, ICML.

[11]  M. Rothschild,et al.  Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets , 1982 .

[12]  P. Bickel,et al.  Covariance regularization by thresholding , 2009, 0901.3079.

[13]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[14]  Gary Chamberlain,et al.  FUNDS, FACTORS, AND DIVERSIFICATION IN ARBITRAGE PRICING MODELS , 1983 .

[15]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[16]  J. Bouchaud,et al.  Noise Dressing of Financial Correlation Matrices , 1998, cond-mat/9810255.

[17]  R. Cook,et al.  On the mean and variance of the generalized inverse of a singular Wishart matrix , 2011 .

[18]  Sheridan Titman,et al.  On Persistence in Mutual Fund Performance , 1997 .

[19]  Christoffer Bengtsson.  On Portfolio Selection : Improved Covariance Matrix Estimation for Swedish Asset Returns , 2002 .

[20]  R. Jagannathan,et al.  Risk Reduction in Large Portfolios: Why Imposing the Wrong Constraints Helps , 2002 .

[21]  David Disatnik,et al.  Shrinking the Covariance Matrix — Simpler is Better Background , 2006 .

[22]  Pradeep Ravikumar,et al.  Sparse inverse covariance matrix estimation using quadratic approximation , 2011, MLSLP.

[23]  T. Cai,et al.  A Constrained ℓ1 Minimization Approach to Sparse Precision Matrix Estimation , 2011, 1102.2233.

[24]  J. Bouchaud,et al.  Theory of Financial Risk and Derivative Pricing: From Statistical Physics to Risk Management , 2011 .

[25]  K. Strimmer,et al.  Statistical Applications in Genetics and Molecular Biology A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics , 2011 .

[26]  F. Nogales,et al.  Size Matters: Optimal Calibration of Shrinkage Estimators for Portfolio Selection , 2011 .

[27]  R. Muirhead.  Aspects of Multivariate Statistical Theory , 1982, Wiley Series in Probability and Statistics.

[28]  J. Lewellen.  The Cross Section of Expected Stock Returns , 2014 .

[29]  Clifford S. Stein.  Estimation of a covariance matrix , 1975 .

[30]  Heng Lian.  Shrinkage tuning parameter selection in precision matrices estimation , 2009 .

[31]  Weidong Liu,et al.  Adaptive Thresholding for Sparse Covariance Matrix Estimation , 2011, 1102.2237.

[32]  Jianqing Fan,et al.  Large covariance estimation by thresholding principal orthogonal complements , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[33]  L. R. Haff.  ESTIMATION OF THE INVERSE COVARIANCE MATRIX: RANDOM MIXTURES OF THE INVERSE WISHART MATRIX AND THE IDENTITY , 1979 .

[34]  Simon Benninga,et al.  Shrinking the Covariance Matrix , 2007 .

[35]  V. Marčenko,et al.  DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES , 1967 .

[36]  Victor DeMiguel,et al.  Optimal Versus Naive Diversification: How Inefficient is the 1/N Portfolio Strategy? , 2009 .

[37]  Olivier Ledoit,et al.  Nonlinear Shrinkage Estimation of Large-Dimensional Covariance Matrices , 2011, 1207.5322.

[38]  Alfred O. Hero,et al.  Shrinkage Algorithms for MMSE Covariance Estimation , 2009, IEEE Transactions on Signal Processing.

[39]  Yan Xu,et al.  Improving Mean Variance Optimization through Sparse Hedging Restrictions , 2013, Journal of Financial and Quantitative Analysis.

[40]  S. Péché,et al.  Eigenvectors of some large sample covariance matrix ensembles , 2009 .

[41]  Walt J. Woerheide.  An index of portfolio diversification , 1993 .

[42]  A. U.S.,et al.  Sparse Estimation of a Covariance Matrix , 2010 .

[43]  Seung-Jean Kim,et al.  Maximum Likelihood Covariance Estimation with a Condition Number Constraint , 2006, 2006 Fortieth Asilomar Conference on Signals, Systems and Computers.

[44]  Yannick Malevergne,et al.  Collective origin of the coexistence of apparent random matrix theory noise and of factors in large sample correlation matrices , 2002, cond-mat/0210115.

[45]  Adam J. Rothman.  Positive definite estimators of large covariance matrices , 2012 .

[46]  Richard O. Michaud.  The Markowitz Optimization Enigma: Is 'Optimized' Optimal? , 1989 .

[47]  V. Siskind.  Second moments of inverse Wishart-matrix elements , 1972 .

[48]  Mirai Tanaka,et al.  Positive definite matrix approximation with condition number constraint , 2014, Optim. Lett..

[49]  Harry M. Markowitz,et al.  The Theory and Practice of Investment Management: Asset Allocation, Valuation, Portfolio Construction, and Strategies , 2011 .

[50]  Bin Yu,et al.  High-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence , 2008, 0811.3628.

[51]  Jianqing Fan,et al.  High Dimensional Covariance Matrix Estimation in Approximate Factor Models , 2011, Annals of statistics.

[52]  L. R. Haff.  The Variational Form of Certain Bayes Estimators , 1991 .

[53]  Vwani P. Roychowdhury,et al.  Covariance selection for nonchordal graphs via chordal embedding , 2008, Optim. Methods Softw..

[54]  Raphael N. Markellos,et al.  Parameter Uncertainty in Portfolio Selection: Shrinking the Inverse Covariance Matrix , 2011 .

[55]  J. Neyman,et al.  INADMISSIBILITY OF THE USUAL ESTIMATOR FOR THE MEAN OF A MULTIVARIATE NORMAL DISTRIBUTION , 2005 .

[56]  Adam J. Rothman,et al.  Generalized Thresholding of Large Covariance Matrices , 2009 .

[57]  Matthew R. McKay,et al.  A Robust Statistics Approach to Minimum Variance Portfolio Optimization , 2015, IEEE Transactions on Signal Processing.

[58]  Eugene P. Wigner,et al.  On a Class of Analytic Functions from the Quantum Theory of Collisions , 1951 .

[59]  Olivier Ledoit,et al.  Improved estimation of the covariance matrix of stock returns with an application to portfolio selection , 2003 .

[60]  A. Stuart,et al.  Portfolio Selection: Efficient Diversification of Investments , 1959 .