Complex Factor Analysis and Extensions

Many subspace-based array signal processing algorithms assume that the noise is spatially white. In this case, the noise covariance matrix is a multiple of the identity and the eigenvectors of the data covariance matrix are not affected by it. If the noise covariance is an unknown arbitrary diagonal (e.g., for an uncalibrated array), the eigenvalue decomposition leads to incorrect subspace estimates and it has to be replaced by a more general “factor analysis” decomposition (FA), which then reveals all relevant information. We consider this data model and several extensions where the noise covariance matrix has a more general structure, such as banded, sparse, block diagonal, and cases, where we have multiple data covariance matrices that share the same noise covariance matrix. Starting from a nonlinear weighted least squares formulation, we propose new estimation algorithms for both classical FA and its extensions. The optimization is based on Gauss–Newton gradient descent. Generally, this leads to an iteration involving the inversion of a very large matrix. Using the structure of the problem, we show how this can be reduced to the inversion of a matrix with dimension equal to the number of unknown noise covariance parameters. This results in new algorithms that have faster numerical convergence and lower complexity compared to several maximum-likelihood based algorithms that could be considered state of the art. The new algorithms scale well to large dimensions and can replace eigenvalue decompositions in many applications even if the noise can be assumed to be white.

[1]  Alle-Jan van der Veen,et al.  Calibration challenges for future radio telescopes , 2010, IEEE Signal Processing Magazine.

[2]  The Gauss-Newton Algorithm for the Weighted Least Squares Factor Analysis , 1978 .

[3]  Petre Stoica,et al.  Maximum-likelihood bearing estimation with partly calibrated arrays in spatially correlated noise fields , 1996, IEEE Trans. Signal Process..

[4]  J. C. Gower,et al.  Factor Analysis as a Statistical Method. 2nd ed. , 1972 .

[5]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[6]  D. Lawley VI.—The Estimation of Factor Loadings by the Method of Maximum Likelihood , 1940 .

[7]  Albert-Jan Boonstra,et al.  Gain calibration methods for radio telescope arrays , 2003, IEEE Trans. Signal Process..

[8]  T. Rothenberg Identification in Parametric Models , 1971 .

[9]  K. Jöreskog A general approach to confirmatory maximum likelihood factor analysis , 1969 .

[10]  C. Anderson‐Cook,et al.  An Introduction to Multivariate Statistical Analysis (3rd ed.) (Book) , 2004 .

[11]  Amir Leshem,et al.  Array Signal Processing for Radio Astronomy , 2004, Experimental Astronomy.

[12]  K. Jöreskog Some contributions to maximum likelihood factor analysis , 1967 .

[13]  S. Kay Fundamentals of statistical signal processing: estimation theory , 1993 .

[14]  Albert-Jan Boonstra,et al.  Spatial Filtering of RF Interference in Radio Astronomy Using a Reference Antenna Array , 2002, IEEE Transactions on Signal Processing.

[15]  Venkatesh Nagesha,et al.  Maximum likelihood estimation for array processing in colored noise , 1996, IEEE Trans. Signal Process..

[16]  Thomas Kailath,et al.  ESPRIT-estimation of signal parameters via rotational invariance techniques , 1989, IEEE Trans. Acoust. Speech Signal Process..

[17]  C. Spearman The proof and measurement of association between two things. , 2015, International journal of epidemiology.

[18]  Alexander Shapiro,et al.  Identifiability of factor analysis: Some results and open problems , 1985 .

[19]  Walter Ledermann,et al.  I.—On a Problem concerning Matrices with Variable Diagonal Elements. , 1940 .

[20]  Anthony J. Weiss,et al.  Detection and localization in colored noise via generalized least squares , 1996, IEEE Trans. Signal Process..

[21]  A. Shapiro Weighted minimum trace factor analysis , 1982 .

[22]  A. Hjørungnes Complex-Valued Matrix Derivatives: Applications in Signal Processing and Communications , 2011 .

[23]  George A. Marcoulides,et al.  TEACHER'S CORNER: Examining Identification Issues in Factor Analysis , 2006 .

[24]  Björn E. Ottersten,et al.  Covariance Matching Estimation Techniques for Array Signal Processing Applications , 1998, Digit. Signal Process..

[25]  A. Goldberger,et al.  Factor analysis by generalized least squares , 1972 .

[26]  Alle-Jan van der Veen,et al.  Multisource Self-Calibration for Sensor Arrays , 2009, IEEE Transactions on Signal Processing.

[27]  Herman Rubin,et al.  Statistical Inference in Factor Analysis , 1956 .

[28]  A. Basilevsky,et al.  Factor Analysis as a Statistical Method. , 1964 .

[29]  Y. Selen,et al.  Model-order selection: a review of information criterion rules , 2004, IEEE Signal Processing Magazine.

[30]  Ralph Otto Schmidt,et al.  A signal subspace approach to multiple emitter location and spectral estimation , 1981 .

[31]  Pablo A. Parrilo,et al.  Diagonal and Low-Rank Matrix Decompositions, Correlation Matrices, and Ellipsoid Fitting , 2012, SIAM J. Matrix Anal. Appl..

[32]  Alle-Jan van der Veen,et al.  Subspace estimation using factor analysis , 2012, 2012 IEEE 7th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[33]  Jianhua Zhao,et al.  ML estimation for factor analysis: EM or non-EM? , 2008, Stat. Comput..

[34]  T. W. Anderson An Introduction to Multivariate Statistical Analysis , 1959 .

[35]  Are Hjrungnes,et al.  Complex-Valued Matrix Derivatives: With Applications in Signal Processing and Communications , 2011 .

[36]  Björn E. Ottersten,et al.  Array processing in correlated noise fields based on instrumental variables and subspace fitting , 1995, IEEE Trans. Signal Process..

[37]  Michael A. Saunders,et al.  MINRES-QLP: A Krylov Subspace Method for Indefinite or Singular Symmetric Systems , 2010, SIAM J. Sci. Comput..

[38]  L. Scharf,et al.  Statistical Signal Processing of Complex-Valued Data: Notation , 2010 .

[39]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[40]  Dorothy T. Thayer,et al.  EM algorithms for ML factor analysis , 1982 .

[41]  Alle-Jan van der Veen,et al.  Application of Krylov based methods in calibration for radio astronomy , 2014, 2014 IEEE 8th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[42]  A. Shapiro Rank-reducibility of a symmetric matrix and sampling theory of minimum trace factor analysis , 1982 .

[43]  S. Markoff,et al.  LOFAR - low frequency array , 2006 .

[44]  A. Seghouane An iterative projections algorithm for ML factor analysis , 2008, 2008 IEEE Workshop on Machine Learning for Signal Processing.