Estimation of the number of spikes, possibly equal, in the high-dimensional case

Estimating the number of spikes in a spiked model is an important problem in many areas such as signal processing. Most of the classical approaches assume a large sample size n whereas the dimension p of the observations is kept small. In this paper, we consider the case of high dimension, where p is large compared to n. The approach is based on recent results of random matrix theory. We extend our previous results to a more difficult situation where some spikes are equal, and compare our algorithm to an existing benchmark method.

[1]  Z. Bai,et al.  Central limit theorems for eigenvalues in a spiked population model , 2008, 0806.2503.

[2]  Victor Solo,et al.  Dimension Estimation in Noisy PCA With SURE and Random Matrix Theory , 2008, IEEE Transactions on Signal Processing.

[3]  Boaz Nadler,et al.  Nonparametric Detection of Signals by Information Theoretic Criteria: Performance Analysis and an Improved Estimator , 2010, IEEE Transactions on Signal Processing.

[4]  Damien Passemier,et al.  On determining the number of spikes in a high-dimensional spiked population model , 2011, 1104.2677.

[5]  Eric R. Ziegel,et al.  Tsukuba Meeting: Largest Attendance Ever , 2004, Technometrics.

[6]  J. Schmee.  An Introduction to Multivariate Statistical Analysis , 1986 .

[7]  Boaz Nadler,et al.  Non-Parametric Detection of the Number of Signals: Hypothesis Testing and Random Matrix Theory , 2009, IEEE Transactions on Signal Processing.

[8]  Clifford Lam,et al.  Factor modeling for high-dimensional time series: inference for the number of factors , 2012, 1206.0613.

[9]  A. Guionnet,et al.  Fluctuations of the Extreme Eigenvalues of Finite Rank Deformations of Random Matrices , 2010, 1009.0145.

[10]  M. Rothschild,et al.  Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets , 1982 .

[11]  Hagit Messer,et al.  Submitted to Ieee Transactions on Signal Processing Detection of Signals by Information Theoretic Criteria: General Asymptotic Performance Analysis , 2022 .

[12]  Alan Edelman,et al.  Sample Eigenvalue Based Detection of High-Dimensional Signals in White Noise Using Relatively Few Samples , 2007, IEEE Transactions on Signal Processing.

[13]  Matthew Harding,et al.  Estimating the Number of Factors in Large Dimensional Factor Models 1 , 2013 .

[14]  I. Johnstone.  On the distribution of the largest eigenvalue in principal components analysis , 2001 .

[15]  S. Ross.  The arbitrage theory of capital asset pricing , 1976 .

[16]  Richard M. Everson,et al.  Inferring the eigenvalues of covariance matrices from limited, noisy data , 2000, IEEE Trans. Signal Process..

[17]  Clifford Lam,et al.  Estimation of latent factors for high-dimensional time series , 2011 .

[18]  M. Hallin,et al.  The Generalized Dynamic-Factor Model: Identification and Estimation , 2000, Review of Economics and Statistics.

[19]  Tormod Næs,et al.  A user-friendly guide to multivariate calibration and classification , 2002 .

[20]  J. W. Silverstein,et al.  Eigenvalues of large sample covariance matrices of spiked population models , 2004, math/0408165.

[21]  Patrick L. Combettes,et al.  Signal detection via spectral theory of large dimensional random matrices , 1992, IEEE Trans. Signal Process..

[22]  Nicolas Lenskyj,et al.  Gary , 2015, The Medical journal of Australia.

[23]  A. Onatski.  TESTING HYPOTHESES ABOUT THE NUMBER OF FACTORS IN LARGE FACTOR MODELS , 2009 .

[24]  Thomas Kailath,et al.  Detection of signals by information theoretic criteria , 1985, IEEE Trans. Acoust. Speech Signal Process..

[25]  Anja Vogler,et al.  An Introduction to Multivariate Statistical Analysis , 2004 .

[26]  B. Nadler,et al.  Determining the number of components in a factor model from limited noisy data , 2008 .

[27]  M. Rothschild,et al.  Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets , 1983 .

[28]  D. Paul.  ASYMPTOTICS OF SAMPLE EIGENSTRUCTURE FOR A LARGE DIMENSIONAL SPIKED COVARIANCE MODEL , 2007 .