Statistical eigen-inference from large Wishart matrices

We consider settings where the observations are drawn from a zero-mean multivariate (real or complex) normal distribution with the population covariance matrix having eigenvalues of arbitrary multiplicity. We assume that the eigenvectors of the population covariance matrix are unknown and focus on inferential procedures that are based on the sample eigenvalues alone (i.e., "eigen-inference"). Results found in the literature establish the asymptotic normality of the fluctuation in the trace of powers of the sample covariance matrix. We develop concrete algorithms for analytically computing the limiting quantities and the covariance of the fluctuations. We exploit the asymptotic normality of the trace of powers of the sample covariance matrix to develop eigenvalue-based procedures for testing and estimation. Specifically, we formulate a simple test of hypotheses for the population eigenvalues and a technique for estimating the population eigenvalues in settings where the cumulative distribution function of the (nonrandom) population eigenvalues has a staircase structure. Monte Carlo simulations are used to demonstrate the superiority of the proposed methodologies over classical techniques and the robustness of the proposed techniques in high-dimensional, (relatively) small sample size settings. The improved performance results from the fact that the proposed inference procedures are "global" (in a sense that we describe) and exploit "global" information thereby overcoming the inherent biases that cripple classical inference procedures which are "local" and rely on "local" information.

[1]  J. W. Silverstein,et al.  Spectral Analysis of Large Dimensional Random Matrices , 2009 .

[2]  M. Srivastava Multivariate Theory for Analyzing High Dimensional Data , 2007 .

[3]  Noureddine El Karoui,et al.  Tracy–Widom limit for the largest eigenvalue of a large class of complex sample covariance matrices , 2005, math/0503109.

[4]  Alan Edelman,et al.  MOPS: Multivariate orthogonal polynomials (symbolically) , 2004, J. Symb. Comput..

[5]  R. Speicher,et al.  Second order freeness and fluctuations of random matrices: II. Unitary random matrices , 2004, math/0405258.

[6]  D. Paul ASYMPTOTICS OF SAMPLE EIGENSTRUCTURE FOR A LARGE DIMENSIONAL SPIKED COVARIANCE MODEL , 2007 .

[7]  H. V. Trees,et al.  Covariance, Subspace, and Intrinsic CramrRao Bounds , 2007 .

[8]  Raj Rao Nadakuditi,et al.  Applied stochastic Eigen-analysis , 2007 .

[9]  M. Srivastava Some tests criteria for the covariance matrix with fewer observations than the dimension , 2006, Acta et commentationes Universitatis Tartuensis de mathematica.

[10]  Noureddine El Karoui,et al.  Spectrum estimation for large dimensional covariance matrices using random matrix theory , 2006, math/0609418.

[11]  Alexandru Nica,et al.  Lectures on the Combinatorics of Free Probability , 2006 .

[12]  Piotr Sniady,et al.  Second order freeness and fluctuations of random matrices. III: Higher order freeness and free cumulants , 2006, Documenta Mathematica.

[13]  Alan Edelman,et al.  Free Probability, Sample Covariance Matrices, and Signal Processing , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[14]  M. Srivastava Some Tests Concerning the Covariance Matrix in High Dimensional Data , 2005 .

[15]  Ronald W. Butler,et al.  Laplace approximations to hypergeometric functions of two matrix arguments , 2005 .

[16]  S.T. Smith,et al.  Covariance, subspace, and intrinsic Crame/spl acute/r-Rao bounds , 2005, IEEE Transactions on Signal Processing.

[17]  O. Zeitouni,et al.  A CLT for a band matrix model , 2004, math/0412040.

[18]  J. W. Silverstein,et al.  Eigenvalues of large sample covariance matrices of spiked population models , 2004, math/0408165.

[19]  R. Speicher,et al.  Second order freeness and fluctuations of random matrices: I. Gaussian and Wishart matrices and cyclic Fock spaces , 2004, math/0405191.

[20]  S. Péché,et al.  Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices , 2004, math/0403022.

[21]  Ioana Dumitriu,et al.  Path Counting and Random Matrix Theory , 2003, Electron. J. Comb..

[22]  Andrew T. A. Wood,et al.  Laplace approximations for hypergeometric functions with matrix argument , 2002 .

[23]  Olivier Ledoit,et al.  Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size , 2002 .

[24]  I. Johnstone On the distribution of the largest eigenvalue in principal components analysis , 2001 .

[25]  J. W. Silverstein,et al.  No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices , 1998 .

[26]  R. Speicher,et al.  On the multiplication of free N-tuples of noncommutative random variables , 1996, funct-an/9604011.

[27]  C. Tracy,et al.  On orthogonal and symplectic matrix ensembles , 1995, solv-int/9509007.

[28]  C. Tracy,et al.  Level-spacing distributions and the Airy kernel , 1992, hep-th/9210074.

[29]  Patrick L. Combettes,et al.  Signal detection via spectral theory of large dimensional random matrices , 1992, IEEE Trans. Signal Process..

[30]  D. Voiculescu Limit laws for Random matrices and free products , 1991 .

[31]  D. Dey,et al.  Estimation of a covariance matrix under Stein's loss , 1985 .

[32]  D. Voiculescu Symmetries of some reduced free product C*-algebras , 1985 .

[33]  R. Muirhead Aspects of Multivariate Statistical Theory , 1982, Wiley Series in Probability and Statistics.

[34]  L. R. Haff Empirical Bayes Estimation of the Multivariate Normal Covariance Matrix , 1980 .

[35]  Thomas L. Marzetta,et al.  Detection, Estimation, and Modulation Theory , 1976 .

[36]  Harry L. Van Trees,et al.  Detection, Estimation, and Modulation Theory, Part I , 1968 .

[37]  T. W. Anderson ASYMPTOTIC THEORY FOR PRINCIPAL COMPONENT ANALYSIS , 1963 .

[38]  J. Wishart THE GENERALISED PRODUCT MOMENT DISTRIBUTION IN SAMPLES FROM A NORMAL MULTIVARIATE POPULATION , 1928 .