Group Symmetric Robust Covariance Estimation

In this paper, we consider Tyler's robust covariance M-estimator under group symmetry constraints. We assume that the covariance matrix is invariant to the conjugation action of a unitary matrix group, referred to as group symmetry. Examples of group symmetric structures include circulant, perHermitian, and proper quaternion matrices. We introduce a group symmetric version of Tyler's estimator (STyler) and provide an iterative fixed point algorithm to compute it. The classical results claim that at least n=p+1 sample points in general position are necessary to ensure the existence and uniqueness of Tyler's estimator, where p is the ambient dimension. We show that the STyler requires significantly less samples. In some groups, even two samples are enough to guarantee its existence and uniqueness. In addition, in the case of elliptical populations, we provide high probability bounds on the error of the STyler. These, too, quantify the advantage of exploiting the symmetry structure. Finally, these theoretical results are supported by numerical simulations.

[1]  Markus Rupp,et al.  Analysis of LMS and NLMS algorithms with delayed coefficient update under the presence of spherically invariant processes , 1994, IEEE Trans. Signal Process..

[2]  J. H. Steiger Tests for comparing elements of a correlation matrix. , 1980 .

[3]  Olivier Besson,et al.  Knowledge-Aided Covariance Matrix Estimation and Adaptive Detection in Compound-Gaussian Noise , 2010, IEEE Transactions on Signal Processing.

[4]  Ali Abdi,et al.  Expected number of maxima in the envelope of a spherically invariant random process , 2003, IEEE Trans. Inf. Theory.

[5]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[6]  Ami Wiesel,et al.  Multivariate Generalized Gaussian Distribution: Convexity and Graphical Models , 2013, IEEE Transactions on Signal Processing.

[7]  Ami Wiesel,et al.  Gaussian graphical models for proper quaternion distributions , 2013, 2013 5th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[8]  Visa Koivunen,et al.  Robust antenna array processing using M-estimators of pseudo-covariance , 2003, 14th IEEE Proceedings on Personal, Indoor and Mobile Radio Communications, 2003. PIMRC 2003..

[9]  O. Reynolds On the dynamical theory of incompressible viscous fluids and the determination of the criterion , 1995, Proceedings of the Royal Society of London. Series A: Mathematical and Physical Sciences.

[10]  Ami Wiesel,et al.  Tyler's Covariance Matrix Estimator in Elliptical Models With Convex Structure , 2014, IEEE Transactions on Signal Processing.

[11]  Regularized covariance estimation in scaled Gaussian models , 2011, 2011 4th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[12]  Yuri I. Abramovich,et al.  Diagonally Loaded Normalised Sample Matrix Inversion (LNSMI) for Outlier-Resistant Adaptive Filtering , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[13]  Alfred O. Hero,et al.  Robust Shrinkage Estimation of High-Dimensional Covariance Matrices , 2010, IEEE Transactions on Signal Processing.

[14]  Giuseppe Ricci,et al.  Performance prediction in compound-Gaussian clutter , 1994 .

[15]  A. Farina,et al.  Radar detection in coherent Weibull clutter , 1987, IEEE Trans. Acoust. Speech Signal Process..

[16]  Joseph A. O'Sullivan,et al.  The use of maximum likelihood estimation for forming images of diffuse radar targets from delay-Doppler data , 1989, IEEE Trans. Inf. Theory.

[17]  C. Curtis,et al.  Representation theory of finite groups and associated algebras , 1962 .

[18]  Matthew R. McKay,et al.  Statistical Linkage Analysis of Substitutions in Patient-Derived Sequences of Genotype 1a Hepatitis C Virus Nonstructural Protein 3 Exposes Targets for Immunogen Design , 2014, Journal of Virology.

[19]  Gabriel Frahm,et al.  Tyler's M-Estimator, Random Matrix Theory, and Generalized Elliptical Distributions with Applications to Finance , 2008 .

[20]  J. Bouchaud,et al.  RANDOM MATRIX THEORY AND FINANCIAL CORRELATIONS , 2000 .

[21]  Teng Zhang Robust subspace recovery by geodesically convex optimization , 2012, 1206.1386.

[22]  SHAUN TAN,et al.  REPRESENTATION THEORY FOR FINITE GROUPS , 2014 .

[23]  Philippe Forster,et al.  Covariance Structure Maximum-Likelihood Estimates in Compound Gaussian Noise: Existence and Algorithm Analysis , 2008, IEEE Transactions on Signal Processing.

[24]  R. Maronna Robust $M$-Estimators of Multivariate Location and Scatter , 1976 .

[25]  Daniel R. Fuhrmann,et al.  Application of Toeplitz covariance estimation to adaptive beamforming and detection , 1991, IEEE Trans. Signal Process..

[26]  On the Distributions of Norms of Spherical Distributions , 2008 .

[27]  Todd M. Allen,et al.  Coordinate linkage of HIV evolution reveals regions of immunological vulnerability , 2011, Proceedings of the National Academy of Sciences.

[28]  H. Vincent Poor,et al.  Complex Elliptically Symmetric Distributions: Survey, New Results and Applications , 2012, IEEE Transactions on Signal Processing.

[29]  M. K. Simon,et al.  Unified theory on wireless communication fading statistics based on SIRP , 2004, IEEE 5th Workshop on Signal Processing Advances in Wireless Communications, 2004..

[30]  J. Billingsley,et al.  Ground Clutter Measurements for Surface-Sited Radar , 1993 .

[31]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[32]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[33]  Colin L. Mallows,et al.  Embedding nonnegative definite Toeplitz matrices in nonnegative definite circulant matrices, with application to covariance estimation , 1989, IEEE Trans. Inf. Theory.

[34]  Ami Wiesel,et al.  Performance Analysis of Tyler's Covariance Estimator , 2014, IEEE Transactions on Signal Processing.

[35]  Xueliang Li,et al.  Eigenvalues of a special kind of symmetric block circulant matrices , 2004 .

[36]  Andrew T. Walden,et al.  Testing for Quaternion Propriety , 2011, IEEE Transactions on Signal Processing.

[37]  Jian Li,et al.  SPICE: A Sparse Covariance-Based Estimation Method for Array Processing , 2011, IEEE Transactions on Signal Processing.

[38]  K. Strimmer,et al.  Statistical Applications in Genetics and Molecular Biology A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics , 2011 .

[39]  Harrison H. Zhou,et al.  Optimal rates of convergence for estimating Toeplitz covariance matrices , 2013 .

[40]  Ami Wiesel,et al.  Geodesic Convexity and Covariance Estimation , 2012, IEEE Transactions on Signal Processing.

[41]  Gabriel Frahm Generalized Elliptical Distributions: Theory and Applications , 2004 .

[42]  Philippe Forster,et al.  Persymmetric Adaptive Radar Detectors , 2011, IEEE Transactions on Aerospace and Electronic Systems.

[43]  David E. Tyler A Distribution-Free $M$-Estimator of Multivariate Scatter , 1987 .

[44]  Jacques Lévy Véhel,et al.  CAPM, Risk and Portfolio Selection in «Stable» Markets , 2000 .

[45]  T. Rapcsák Geodesic convexity in nonlinear optimization , 1991 .

[46]  Ami Wiesel,et al.  Time Varying Autoregressive Moving Average Models for Covariance Estimation , 2013, IEEE Transactions on Signal Processing.

[47]  Nicolas Le Bihan,et al.  Quaternion-MUSIC for vector-sensor array processing , 2006, IEEE Transactions on Signal Processing.

[48]  P. Bickel,et al.  Regularized estimation of large covariance matrices , 2008, 0803.1909.

[49]  V. V. Yurinskii Exponential inequalities for sums of random vectors , 1976 .

[50]  Olivier Ledoit,et al.  Improved estimation of the covariance matrix of stock returns with an application to portfolio selection , 2003 .

[51]  K. Murota,et al.  A numerical algorithm for block-diagonal decomposition of matrix *-algebras with general irreducible components , 2010 .

[52]  Antonio De Maio Maximum likelihood estimation of structured persymmetric covariance matrices , 2003, Signal Process..

[53]  Fulvio Gini,et al.  Cramér-Rao Lower Bounds on Covariance Matrix Estimation for Complex Elliptically Symmetric Distributions , 2013, IEEE Transactions on Signal Processing.

[54]  F. Gini Sub-optimum coherent radar detection in a mixture of K-distributed and Gaussian clutter , 1997 .

[55]  Ami Wiesel,et al.  Unified Framework to Regularized Covariance Estimation in Scaled Gaussian Models , 2012, IEEE Transactions on Signal Processing.

[56]  Yuri I. Abramovich,et al.  Time-Varying Autoregressive (TVAR) Models for Multiple Radar Observations , 2007, IEEE Transactions on Signal Processing.

[57]  D. Pollock,et al.  Circulant matrices and time-series analysis , 2000 .

[58]  Phillipp Meister,et al.  Statistical Signal Processing Detection Estimation And Time Series Analysis , 2016 .

[59]  Simon Watts,et al.  Radar detection prediction in sea clutter using the compound K-distribution model , 1985 .

[60]  David E. Tyler Statistical analysis for the angular central Gaussian distribution on the sphere , 1987 .

[61]  Parikshit Shah,et al.  Group symmetry and covariance regularization , 2011, 2012 46th Annual Conference on Information Sciences and Systems (CISS).