Submitted to the Annals of Statistics TESTING IN HIGH-DIMENSIONAL SPIKED MODELS By

We consider the five classes of multivariate statistical problems identified by James (1964), which together cover much of classical multivariate analysis, plus a simpler limiting case, symmetric matrix denoising. Each of James’ problems involves the eigenvalues of −1 where  and  are proportional to high dimensional Wishart matrices. Under the null hypothesis, both Wisharts are central with identity covariance. Under the alternative, the non-centrality or the covariance parameter of has a single eigenvalue, a spike, that stands alone. When the spike is smaller than a case-specific phase transition threshold, none of the sample eigenvalues separate from the bulk, making the testing problem challenging. Using a unified strategy for the six cases, we show that the log likelihood ratio processes parameterized by the value of the sub-critical spike converge to Gaussian processes with logarithmic correlation. We then derive asymptotic power envelopes for tests for the presence of a spike.

[1]  Prathapasinghe Dharmawansa,et al.  Joint density of eigenvalues in spiked multivariate models , 2014, Stat.

[2]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[3]  Jianfeng Yao,et al.  On the convergence of the spectral empirical process of Wigner matrices , 2005 .

[4]  R. Paris.  Asymptotics of the Gauss hypergeometric function with large parameters, I , 2013 .

[5]  Matthew R. McKay,et al.  Asymptotic Linear Spectral Statistics for Spiked Hermitian Random Matrices , 2014, 1402.6419.

[6]  L. Milne‐Thomson.  A Treatise on the Theory of Bessel Functions , 1945, Nature.

[7]  Andrew B. Nobel,et al.  Reconstruction of a low-rank matrix in the presence of Gaussian noise , 2010, J. Multivar. Anal..

[8]  Ronald F. Boisvert,et al.  NIST Handbook of Mathematical Functions , 2010 .

[9]  J. W. Silverstein,et al.  Eigenvalues of large sample covariance matrices of spiked population models , 2004, math/0408165.

[10]  Raj Rao Nadakuditi,et al.  Fundamental Limit of Sample Generalized Eigenvalue Based Detection of Signals in Noise Using Relatively Few Signal-Bearing and Noise-Only Samples , 2009, IEEE Journal of Selected Topics in Signal Processing.

[11]  Shurong Zheng,et al.  Central limit theorems for linear spectral statistics of large dimensional F-matrices , 2012 .

[12]  Z. Bai,et al.  Convergence to the Semicircle Law , 1988 .

[13]  J. Azaïs,et al.  The Distribution of the Maximum of a Gaussian Process: Rice Method Revisited. , 2002 .

[14]  Marcelo J. Moreira,et al.  Asymptotic power of sphericity tests for high-dimensional data , 2013, 1306.4867.

[15]  J. Baik,et al.  On the largest eigenvalue of a Hermitian random matrix model with spiked external source II. Higher rank cases , 2011, 1104.2915.

[16]  Alexei Onatski,et al.  Signal detection in high dimension: The multispiked case , 2012, 1210.5663.

[17]  Marc Lelarge,et al.  Fundamental limits of symmetric low-rank matrix estimation , 2016, Probability Theory and Related Fields.

[18]  Robb J. Muirhead,et al.  Latent Roots and Matrix Variates: A Review of Some Asymptotic Results , 1978 .

[19]  D. Pollard,et al.  Cube Root Asymptotics , 1990 .

[20]  D. Donoho,et al.  Minimax risk of matrix denoising by singular value thresholding , 2013, 1304.2085.

[21]  F. Olver.  Asymptotics and Special Functions , 1974 .

[22]  M. Abramowitz,et al.  Handbook of Mathematical Functions With Formulas, Graphs and Mathematical Tables (National Bureau of Standards Applied Mathematics Series No. 55) , 1965 .

[23]  A. Onatski.  Asymptotics of the principal components estimator of large factor models with weak factors and i.i.d. Gaussian noise , 2018 .

[24]  Mylène Maïda,et al.  Large deviations for the largest eigenvalue of rank one deformations of Gaussian ensembles , 2006 .

[25]  E. Dobriban,et al.  Sharp detection in PCA under correlations: all eigenvalues matter , 2016, 1602.06896.

[26]  Raj Rao Nadakuditi,et al.  OptShrink: An Algorithm for Improved Low-Rank Signal Matrix Denoising by Optimal, Data-Driven Singular Value Shrinkage , 2013, IEEE Transactions on Information Theory.

[27]  R. Muirhead.  Aspects of Multivariate Statistical Theory , 1982, Wiley Series in Probability and Statistics.

[28]  Z. Bai.  METHODOLOGIES IN SPECTRAL ANALYSIS OF LARGE DIMENSIONAL RANDOM MATRICES , A REVIEW , 1999 .

[29]  J. W. Silverstein,et al.  COVARIANCE MATRICES , 2022 .

[30]  Jiang Hu,et al.  Canonical correlation coefficients of high-dimensional Gaussian vectors: Finite rank case , 2014, The Annals of Statistics.

[31]  J. W. Silverstein,et al.  No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices , 1998 .

[32]  N. Simm,et al.  FRACTIONAL BROWNIAN MOTION WITH HURST INDEX H=0 AND THE GAUSSIAN UNITARY ENSEMBLE , 2013, 1312.0212.

[33]  Alexei Onatski,et al.  Local Asymptotic Normality of the Spectrum of High-Dimensional Spiked F-Ratios , 2014, 1411.3875.

[34]  A. V. D. Vaart.  Asymptotic Statistics: Delta Method , 1998 .

[35]  A. James.  Distributions of Matrix Variates and Latent Roots Derived from Normal Samples , 1964 .