Tuning the Parameters for Precision Matrix Estimation Using Regression Analysis

Precision matrix, i.e., inverse covariance matrix, is widely used in signal processing, and often estimated from training samples. Regularization techniques, such as banding and rank reduction, can be applied to the covariance matrix or precision matrix estimation for improving the estimation accuracy when the training samples are limited. In this paper, exploiting regression interpretations of the precision matrix, we introduce two data-driven, distribution-free methods to tune the parameter for regularized precision matrix estimation. The numerical examples are provided to demonstrate the effectiveness of the proposed methods and example applications in the design of minimum mean squared error (MMSE) channel estimators for large-scale multiple-input multiple-output (MIMO) communication systems are demonstrated.

[1]  Jiangtao Xi,et al.  Linear shrinkage estimation of covariance matrices using low-complexity cross-validation , 2018, Signal Process..

[2]  Olivier Ledoit,et al.  A well-conditioned estimator for large-dimensional covariance matrices , 2004 .

[3]  Alfred O. Hero,et al.  Shrinkage Algorithms for MMSE Covariance Estimation , 2009, IEEE Transactions on Signal Processing.

[4]  Jun Fang,et al.  Low-Rank Covariance-Assisted Downlink Training and Channel Estimation for FDD Massive MIMO Systems , 2016, IEEE Transactions on Wireless Communications.

[5]  Ernst Bonek,et al.  A stochastic MIMO channel model with joint correlation of both link ends , 2006, IEEE Transactions on Wireless Communications.

[6]  P. Bickel,et al.  Covariance regularization by thresholding , 2009, 0901.3079.

[7]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[8]  Gene H. Golub,et al.  Generalized cross-validation as a method for choosing a good ridge parameter , 1979, Milestones in Matrix Computation.

[9]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[10]  Jiangtao Xi,et al.  Shrinkage of Covariance Matrices for Linear Signal Estimation Using Cross-Validation , 2016, IEEE Transactions on Signal Processing.

[11]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[12]  Harrison H. Zhou,et al.  Optimal rates of convergence for covariance matrix estimation , 2010, 1010.3866.

[13]  Xiaohui Chen,et al.  Shrinkage-to-Tapering Estimation of Large Covariance Matrices , 2012, IEEE Transactions on Signal Processing.

[14]  Fei Wen,et al.  Positive Definite Estimation of Large Covariance Matrix Using Generalized Nonconvex Penalties , 2016, IEEE Access.

[15]  Luc Devroye,et al.  Distribution-free performance bounds for potential function rules , 1979, IEEE Trans. Inf. Theory.

[16]  Arjun K. Gupta,et al.  Direct shrinkage estimation of large dimensional precision matrix , 2013, J. Multivar. Anal..

[17]  Yumou Qiu,et al.  Bandwidth Selection for High-Dimensional Covariance Matrix Estimation , 2014 .

[18]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[19]  Don H. Johnson,et al.  Statistical Signal Processing , 2009, Encyclopedia of Biometrics.

[20]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[21]  Lixing Zhu,et al.  SHRINKAGE ESTIMATION OF LARGE DIMENSIONAL PRECISION MATRIX USING RANDOM MATRIX THEORY , 2015 .

[22]  Jianqing Fan,et al.  An Overview of the Estimation of Large Covariance and Precision Matrices , 2015, The Econometrics Journal.

[23]  Jie Zhou,et al.  Estimation of Large Covariance Matrices by Shrinking to Structured Target in Normal and Non-Normal Distributions , 2018, IEEE Access.

[24]  Joseph R. Guerci,et al.  Space-Time Adaptive Processing for Radar , 2003 .

[25]  Sylvain Arlot,et al.  A survey of cross-validation procedures for model selection , 2009, 0907.4728.

[26]  Patrice Abry,et al.  Covariance Versus Precision Matrix Estimation for Efficient Asset Allocation , 2016, IEEE Journal of Selected Topics in Signal Processing.

[27]  Anne H. Schistad Solberg,et al.  Regression Approaches to Small Sample Inverse Covariance Matrix Estimation for Hyperspectral Image Classification , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Y. Selen,et al.  Model-order selection: a review of information criterion rules , 2004, IEEE Signal Processing Magazine.

[29]  Nicholas J. Higham,et al.  Restoring Definiteness via Shrinking, with an Application to Correlation Matrices with a Fixed Block , 2016, SIAM Rev..

[30]  Rina Foygel,et al.  Extended Bayesian Information Criteria for Gaussian Graphical Models , 2010, NIPS.

[31]  Yumou Qiu,et al.  Test for Bandedness of High Dimensional Covariance Matrices with Bandwidth Estimation , 2012, 1208.3321.

[32]  P. Bickel,et al.  Regularized estimation of large covariance matrices , 2008, 0803.1909.

[33]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[34]  Yang Song,et al.  Canonical correlation analysis of high-dimensional data with very small sample support , 2016, Signal Process..

[35]  Jie Ding,et al.  Bridging AIC and BIC: A New Criterion for Autoregression , 2015, IEEE Transactions on Information Theory.

[36]  Jianhua Z. Huang,et al.  Covariance matrix selection and estimation via penalised normal likelihood , 2006 .

[37]  M. Pourahmadi,et al.  Nonparametric estimation of large covariance matrices of longitudinal data , 2003 .

[38]  Jie Ding,et al.  Model Selection Techniques: An Overview , 2018, IEEE Signal Processing Magazine.

[39]  H. Akaike.  A new look at the statistical model identification , 1974 .

[40]  Adam J. Rothman,et al.  A new approach to Cholesky-based covariance regularization in high dimensions , 2009, 0903.0645.

[41]  Montse Nájar,et al.  Asymptotically Optimal Linear Shrinkage of Sample LMMSE and MVDR Filters , 2014, IEEE Transactions on Signal Processing.

[42]  Jiangtao Xi,et al.  Cross-Validated Bandwidth Selection for Precision Matrix Estimation , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[43]  Ernst Wit,et al.  A computationally fast alternative to cross-validation in penalized Gaussian graphical models , 2013, 1309.6216.

[44]  Emil Björnson,et al.  Low-Complexity Polynomial Channel Estimation in Large-Scale MIMO With Arbitrary Statistics , 2014, IEEE Journal of Selected Topics in Signal Processing.

[45]  Joseph M. Kahn,et al.  Fading correlation and its effect on the capacity of multielement antenna systems , 2000, IEEE Trans. Commun..

[46]  Yufeng Liu,et al.  Hypothesis Testing for Band Size Detection of High Dimensional Banded Precision Matrices , 2014, Biometrika.