Asymptotic power of sphericity tests for high-dimensional data

This paper studies the asymptotic power of tests of sphericity against perturbations in a single unknown direction as both the dimensionality of the data and the number of observations go to infinity. We establish the convergence, under the null hypothesis and the alternative, of the log ratio of the joint densities of the sample covariance eigenvalues to a Gaussian process indexed by the norm of the perturbation. When the perturbation norm is larger than the phase transition threshold studied in Baik et al. (2005), the limiting process is degenerate and discrimination between the null and the alternative is asymptotically certain. When the norm is below the threshold, the process is non-degenerate, so that the joint eigenvalue densities under the null and alternative hypotheses are mutually contiguous. Using the asymptotic theory of statistical experiments, we obtain asymptotic power envelopes and derive the asymptotic power for various sphericity tests in the contiguity region. In particular, we show that the asymptotic power of the Tracy-Widom-type tests is trivial, whereas that of the eigenvalue-based likelihood ratio test is strictly larger than the size, and close to the power envelope.

[1]  J. Mauchly.  Significance Test for Sphericity of a Normal $n$-Variate Distribution , 1940 .

[2]  Le Cam,et al.  Locally asymptotically normal families of distributions : certain approximations to families of distributions & thier use in the theory of estimation & testing hypotheses , 1960 .

[3]  A. James.  Distributions of Matrix Variates and Latent Roots Derived from Normal Samples , 1964 .

[4]  W. Rudin.  Real and complex analysis , 1968 .

[5]  S. John.  Some optimal multivariate tests , 1971 .

[6]  N. Sugiura.  Locally Best Invariant Test for Sphericity and the Limiting Distributions , 1972 .

[7]  S. John.  The distribution of a statistic used for testing sphericity of normal distributions , 1972 .

[8]  F. Olver.  Asymptotics and Special Functions , 1974 .

[9]  S. Geman.  A Limit Theorem for the Norm of Random Matrices , 1980 .

[10]  J. Dickey.  Multiple Hypergeometric Functions: Probabilistic Interpretations and Statistical Uses , 1983 .

[11]  P. W. Karlsson,et al.  Multiple Gaussian hypergeometric series , 1985 .

[12]  W. Rudin.  Real and complex analysis, 3rd ed. , 1987 .

[13]  Z. Bai,et al.  Convergence Rate of Expected Spectral Distributions of Large Random Matrices. Part II. Sample Covariance Matrices , 1993 .

[14]  Z. Bai,et al.  Convergence rate of expected spectral distributions of large random matrices , 2008 .

[15]  Z. Bai,et al.  Convergence Rate of Expected Spectral Distributions of Large Random Matrices. Part I. Wigner Matrices , 1993 .

[16]  H. Uhlig.  On singular Wishart and singular multivariate beta distributions , 1994 .

[17]  J. W. Silverstein,et al.  On the empirical distribution of eigenvalues of a class of large dimensional random matrices , 1995 .

[18]  A. V. D. Vaart,et al.  Asymptotic Statistics: U -Statistics , 1998 .

[19]  I. Johnstone.  On the distribution of the largest eigenvalue in principal components analysis , 2001 .

[20]  G. Hillier.  THE DENSITY OF A QUADRATIC FORM IN A VECTOR UNIFORMLY DISTRIBUTED ON THE n-SPHERE , 2001, Econometric Theory.

[21]  Olivier Ledoit,et al.  Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size , 2002 .

[22]  J. Azaïs,et al.  The Distribution of the Maximum of a Gaussian Process: Rice Method Revisited. , 2002 .

[23]  Andrew T. A. Wood,et al.  Laplace approximations for hypergeometric functions with matrix argument , 2002 .

[24]  Holger Dette,et al.  A note on testing the covariance matrix for large dimension , 2005 .

[25]  S. Péché,et al.  Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices , 2004, math/0403022.

[26]  J. W. Silverstein,et al.  Eigenvalues of large sample covariance matrices of spiked population models , 2004, math/0408165.

[27]  Means of a Dirichlet process and multiple hypergeometric functions , 2004, math/0410151.

[28]  T. Ratnarajah,et al.  Complex singular wishart matrices and applications , 2005 .

[29]  M. Srivastava.  Some Tests Concerning the Covariance Matrix in High Dimensional Data , 2005 .

[30]  A. Guionnet,et al.  A Fourier view on the R-transform and related asymptotics of spherical integrals , 2005 .

[31]  D. Reich,et al.  Population Structure and Eigenanalysis , 2006, PLoS genetics.

[32]  D. Paindaveine,et al.  SEMIPARAMETRICALLY EFFICIENT RANK-BASED INFERENCE FOR SHAPE I. OPTIMAL RANK-BASED TESTS FOR SPHERICITY , 2006, 0707.4621.

[33]  James R. Schott,et al.  A high-dimensional test for the equality of the smallest eigenvalues of a covariance matrix , 2006 .

[34]  A. Bejan,et al.  LARGEST EIGENVALUES AND SAMPLE COVARIANCE MATRICES. TRACY-WIDOM AND PAINLEVÉ II: COMPUTATIONAL ASPECTS AND REALIZATION IN S-PLUS WITH APPLICATIONS , 2006 .

[35]  J. W. Silverstein,et al.  Fundamental Limit of Sample Eigenvalue based Detection of Signals in Colored Noise using Relatively Few Samples , 2007, 2007 Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers.

[36]  Noureddine El Karoui.  Tracy–Widom limit for the largest eigenvalue of a large class of complex sample covariance matrices , 2005, math/0503109.

[37]  Raymond Kan,et al.  COMPUTATIONALLY EFFICIENT RECURSIONS FOR TOP-ORDER INVARIANT POLYNOMIALS WITH APPLICATIONS , 2009, Econometric Theory.

[38]  Alan Edelman,et al.  Sample Eigenvalue Based Detection of High-Dimensional Signals in White Noise Using Relatively Few Samples , 2007, IEEE Transactions on Signal Processing.

[39]  Z. Bai,et al.  METHODOLOGIES IN SPECTRAL ANALYSIS OF LARGE DIMENSIONAL RANDOM MATRICES, A REVIEW , 2008 .

[40]  B. Nadler,et al.  Determining the number of components in a factor model from limited noisy data , 2008 .

[41]  David C. Hoyle,et al.  Automatic PCA Dimension Selection for High Dimensional Data and Small Sample Sizes , 2008 .

[42]  B. Nadler.  Finite sample approximation results for principal component analysis: a matrix perturbation approach , 2009, 0901.3245.

[43]  A. Onatski.  TESTING HYPOTHESES ABOUT THE NUMBER OF FACTORS IN LARGE FACTOR MODELS , 2009 .

[44]  Z. Bai,et al.  Corrections to LRT on large-dimensional covariance matrix by RMT , 2009, 0902.0552.

[45]  S. P'ech'e,et al.  The largest eigenvalues of sample covariance matrices for a spiked population: Diagonal case , 2008, 0812.2320.

[46]  Boaz Nadler,et al.  Non-Parametric Detection of the Number of Signals: Hypothesis Testing and Random Matrix Theory , 2009, IEEE Transactions on Signal Processing.

[47]  Raj Rao Nadakuditi,et al.  Fundamental Limit of Sample Generalized Eigenvalue Based Detection of Signals in Noise Using Relatively Few Signal-Bearing and Noise-Only Samples , 2009, IEEE Journal of Selected Topics in Signal Processing.

[48]  A. Onatski.  Determining the Number of Factors from Empirical Distribution of Eigenvalues , 2010, The Review of Economics and Statistics.

[49]  Zhidong Bai,et al.  CLT for Linear Spectral Statistics , 2010 .

[50]  Patrick J. Wolfe,et al.  Minimax Rank Estimation for Subspace Tracking , 2009, IEEE Journal of Selected Topics in Signal Processing.

[51]  Song-xi Chen,et al.  Tests for High-Dimensional Covariance Matrices , 2010, Random Matrices: Theory and Applications.

[52]  Xiaoqian Sun,et al.  A new test for sphericity of the covariance matrix for high dimensional data , 2010, J. Multivar. Anal..

[53]  M. Y. Mo.  Rank 1 real Wishart spiked model , 2011, 1101.5144.

[54]  Pascal Bianchi,et al.  Performance of Statistical Tests for Single-Source Detection Using Random Matrix Theory , 2009, IEEE Transactions on Information Theory.

[55]  Alex Bloemendal,et al.  Limits of spiked random matrices II , 2011, 1109.3704.

[56]  Dong Wang.  The Largest Eigenvalue of Real Symmetric, Hermitian and Hermitian Self-dual Random Matrix Models with Rank One External Source, Part I , 2010, 1012.4144.

[57]  P. Rigollet,et al.  Optimal detection of sparse principal components in high dimension , 2012, 1202.5070.

[58]  Alexei Onatski,et al.  Asymptotics of the principal components estimator of large factor models with weakly influential factors , 2012 .

[59]  Alex Bloemendal,et al.  Limits of spiked random matrices I , 2010, Probability Theory and Related Fields.