Bayesian inference for PCA and MUSIC algorithms with unknown number of sources

Principal component analysis (PCA) is a popular method for projecting data onto uncorrelated components in lower dimension, although the optimal number of components is not specified. Likewise, multiple signal classification (MUSIC) algorithm is a popular PCA-based method for estimating directions of arrival (DOAs) of sinusoidal sources, yet it requires the number of sources to be known a priori. The accurate estimation of the number of sources is hence a crucial issue for performance of these algorithms. In this paper, we will show that both PCA and MUSIC actually return the exact joint maximum-a-posteriori (MAP) estimate for uncorrelated steering vectors, although they can only compute this MAP estimate approximately in correlated case. We then use Bayesian method to, for the first time, compute the MAP estimate for the number of sources in PCA and MUSIC algorithms. Intuitively, this MAP estimate corresponds to the highest probability that signal-plus-noise's variance still dominates projected noise's variance on signal subspace. In simulations of overlapping multi-tone sources for linear sensor array, our exact MAP estimate is far superior to the asymptotic Akaike information criterion (AIC), which is a popular method for estimating the number of components in PCA and MUSIC algorithms.

[1]  Gongguo Tang,et al.  Near minimax line spectral estimation , 2013, 2013 47th Annual Conference on Information Sciences and Systems (CISS).

[2]  Nicolas Dobigeon,et al.  Bayesian nonparametric subspace estimation , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[4]  Benjamin Recht,et al.  Atomic norm denoising with applications to line spectral estimation , 2011, Allerton.

[5]  David L. Donoho,et al.  The Optimal Hard Threshold for Singular Values is 4/sqrt(3) , 2013, 1305.5870.

[6]  Zhihua Zhang,et al.  EP-GIG Priors and Applications in Bayesian Sparse Learning , 2012, J. Mach. Learn. Res..

[7]  Jean-Yves Tourneret,et al.  Minimum Mean Square Distance Estimation of a Subspace , 2011, IEEE Transactions on Signal Processing.

[8]  Václav Smídl,et al.  On Bayesian principal component analysis , 2007, Comput. Stat. Data Anal..

[9]  Viet Hung Tran,et al.  Variational Bayes Inference in Digital Receivers , 2018, ArXiv.

[10]  G. Camps-Valls,et al.  A Support Vector Machine MUSIC Algorithm , 2012, IEEE Transactions on Antennas and Propagation.

[11]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[12]  H. Araki,et al.  An inequality for Hilbert-Schmidt norm , 1981 .

[13]  Daniel T. Larose,et al.  Discovering Knowledge in Data: An Introduction to Data Mining , 2005 .

[14]  Inder J. Gupta,et al.  A Modified MUSIC Algorithm for Direction of Arrival Estimation in the Presence of Antenna Array Manifold Mismatch , 2016, IEEE Transactions on Antennas and Propagation.

[15]  Lihua Xie,et al.  Enhancing Sparsity and Resolution via Reweighted Atomic Norm Minimization , 2014, IEEE Transactions on Signal Processing.

[16]  Arye Nehorai,et al.  Improved Source Number Detection and Direction Estimation With Nested Arrays and ULAs Using Jackknifing , 2013, IEEE Transactions on Signal Processing.

[17]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[18]  V. K. Kokate,et al.  Performance Analysis of MUSIC and ESPRIT DOA Estimation Algorithms for Adaptive Array Smart Antenna in Mobile Communication , 2010, 2010 Second International Conference on Computer and Network Technology.

[19]  H. Akaike.  A new look at the statistical model identification , 1974 .

[20]  Lyudmila Mihaylova,et al.  Bayesian Compressive Sensing Approaches for Direction of Arrival Estimation With Mutual Coupling Effects , 2017, IEEE Transactions on Antennas and Propagation.

[21]  Tom Minka,et al.  Automatic Choice of Dimensionality for PCA , 2000, NIPS.

[22]  Jisheng Dai,et al.  Root Sparse Bayesian Learning for Off-Grid DOA Estimation , 2016, IEEE Signal Processing Letters.

[23]  Bernard H. Fleury,et al.  Variational Bayesian Inference of Line Spectra , 2016, IEEE Transactions on Signal Processing.

[24]  Jorge Cadima,et al.  Principal component analysis: a review and recent developments , 2016, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[25]  Oliver Henkel,et al.  Sphere-packing bounds in the Grassmann and Stiefel manifolds , 2003, IEEE Transactions on Information Theory.

[26]  Jim Freeman.  Probability Metrics and the Stability of Stochastic Models , 1991 .

[27]  Jonathon A. Chambers,et al.  Bayesian Inference for Multi-Line Spectra in Linear Sensor Array , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[28]  Tariq Pervez Sattar,et al.  Comparison between MUSIC and ESPRIT direction of arrival estimation algorithms for wireless communication systems , 2012, The First International Conference on Future Generation Communication Technologies.

[29]  Pierre-Alexandre Mattei,et al.  Model selection for sparse high-dimensional learning , 2017 .

[30]  Clement Elvira.  Bayesian models for anti-sparse coding and Bayesian non parametric principal component analysis. (Modèles bayésiens pour l'identification de représentations antiparcimonieuses et l'analyse en composantes principales bayésienne non paramétrique) , 2017 .

[31]  Hagit Messer,et al.  Submitted to Ieee Transactions on Signal Processing Detection of Signals by Information Theoretic Criteria: General Asymptotic Performance Analysis , 2022 .

[32]  Christoph F. Mecklenbräuker,et al.  Sequential Bayesian Sparse Signal Reconstruction Using Array Data , 2013, IEEE Transactions on Signal Processing.

[33]  Malene Højbjerre,et al.  Linear and Graphical Models , 1995 .