Partial Coherence Estimation via Spectral Matrix Shrinkage under Quadratic Loss

Partial coherence is an important quantity derived from spectral or precision matrices and is used in seismology, meteorology, oceanography, neuroscience and elsewhere. If the number of complex degrees of freedom only slightly exceeds the dimension of the multivariate stationary time series, spectral matrices are poorly conditioned and shrinkage techniques suggest themselves. When true partial coherencies are quite large, then for shrinkage estimators of the diagonal weighting kind, it is shown empirically that the minimization of risk using quadratic loss (QL) leads to oracle partial coherence estimators far superior to those derived by minimizing risk using Hilbert-Schmidt (HS) loss. When true partial coherencies are small, the methods behave similarly. We derive two new QL estimators for spectral matrices, and new QL and HS estimators for precision matrices. In addition, for the full estimation (non-oracle) case where certain trace expressions must also be estimated, we examine the behavior of three different QL estimators, with the precision matrix one seeming particularly appealing. For the empirical study, we carry out exact simulations derived from real EEG data for two individuals, one having large, and the other small, partial coherencies. This ensures that our study covers cases of real-world relevance.

[1]  Olivier Ledoit,et al.  A well-conditioned estimator for large-dimensional covariance matrices , 2004 .

[2]  A. Walden,et al.  Brain connectivity in positive and negative syndrome schizophrenia , 2010, Neuroscience.

[3]  C. Stein,et al.  Estimation with Quadratic Loss , 1992 .

[4]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[5]  Gérard Letac,et al.  All Invariant Moments of the Wishart Distribution , 2004 .

[6]  G L Gebber,et al.  Partial spectral analysis of cardiac-related sympathetic nerve discharge. , 2000, Journal of neurophysiology.

[7]  Xiaohui Chen,et al.  Shrinkage-to-Tapering Estimation of Large Covariance Matrices , 2012, IEEE Transactions on Signal Processing.

[8]  E. Bullmore,et al.  Undirected graphs of frequency-dependent functional connectivity in whole brain networks , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[9]  Andrew T. Walden,et al.  Random Matrix Derived Shrinkage of Spectral Precision Matrices , 2014, IEEE Transactions on Signal Processing.

[10]  Clifford S. Stein.  Estimation of a covariance matrix , 1975 .

[11]  Donald B. Percival,et al.  Spectral Analysis for Physical Applications , 1993 .

[12]  Richard L. Smith,et al.  Essentials of statistical inference , 2005 .

[13]  Andrew T. Walden,et al.  Statistical Properties of the Estimator of the Rotary Coefficient , 2011, IEEE Transactions on Signal Processing.

[14]  R. Dahlhaus.  Graphical interaction models for multivariate time series1 , 2000 .

[15]  R.E. White,et al.  Signal and noise estimation from seismic reflection data using spectral coherence methods , 1984, Proceedings of the IEEE.

[16]  D. Booth,et al.  Descriptive Physical Oceanography of Great-Entry Lagoon, Gulf of St. Lawrence , 2002 .

[17]  J. Neyman.  Contributions to the theory of statistics , 1956 .

[18]  Kurt S. Riedel,et al.  Minimum bias multiple taper spectral estimation , 2018, IEEE Trans. Signal Process..

[19]  Andrew T. Walden,et al.  Simulation Methodology for Inference on Physical Parameters of Complex Vector-Valued Signals , 2013, IEEE Transactions on Signal Processing.

[20]  Hernando Ombao,et al.  Functional connectivity: Shrinkage estimation and randomization test , 2010, NeuroImage.

[21]  Yoshihiko Konno,et al.  Shrinkage estimators for large covariance matrices in multivariate real and complex normal distributions under an invariant quadratic loss , 2009, J. Multivar. Anal..

[22]  Lixing Zhu,et al.  SHRINKAGE ESTIMATION OF LARGE DIMENSIONAL PRECISION MATRIX USING RANDOM MATRIX THEORY , 2015 .

[23]  B. Efron,et al.  Multivariate Empirical Bayes and Estimation of Covariance Matrices , 1976 .

[24]  Donald B. Percival,et al.  The effective bandwidth of a multitaper spectral estimator , 1995 .

[25]  L. R. Haff.  Empirical Bayes Estimation of the Multivariate Normal Covariance Matrix , 1980 .

[26]  B. Carlson.  Covariance matrix estimation errors and diagonal loading in adaptive arrays , 1988 .

[27]  A. T. Walden,et al.  Graphical modelling for brain connectivity via partial coherence , 2009, Journal of Neuroscience Methods.

[28]  D. Kraus,et al.  Calculation of moments of complex Wishart and complex inverse Wishart distributed matrices , 2000 .

[29]  Robert L. Wolpert,et al.  Statistical Inference , 2019, Encyclopedia of Social Network Analysis and Mining.

[30]  N. R. Goodman.  Statistical analysis based on a certain multivariate complex Gaussian distribution , 1963 .

[31]  D. Harville.  Matrix Algebra From a Statistician's Perspective , 1998 .

[32]  L. R. Haff.  ESTIMATION OF THE INVERSE COVARIANCE MATRIX: RANDOM MIXTURES OF THE INVERSE WISHART MATRIX AND THE IDENTITY , 1979 .

[33]  R. Kass,et al.  Shrinkage Estimators for Covariance Matrices , 2001, Biometrics.

[34]  G. J. Fraser,et al.  The 5-day wave and ionospheric absorption , 1977 .

[35]  M. Hallett,et al.  Functional coupling of human right and left cortical motor areas demonstrated with partial coherence analysis , 2000, Neuroscience Letters.

[36]  K. Strimmer,et al.  Statistical Applications in Genetics and Molecular Biology A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics , 2011 .

[37]  Richard L. Smith,et al.  Essentials of Statistical Inference: Index , 2005 .

[38]  Xiaoqian Sun,et al.  Improved Stein-type shrinkage estimators for the high-dimensional multivariate normal covariance matrix , 2011, Comput. Stat. Data Anal..