Measure-Transformed Quasi-Maximum Likelihood Estimation

In this paper, the Gaussian quasi-maximum likelihood estimator (GQMLE) is generalized by applying a transform to the probability distribution of the data. The proposed estimator, called measure-transformed GQMLE (MT-GQMLE), minimizes the empirical Kullback-Leibler divergence between a transformed probability distribution of the data and a hypothesized Gaussian probability measure. By judicious choice of the transform we show that, unlike the GQMLE, the proposed estimator can gain sensitivity to higher order statistical moments and resilience to outliers leading to significant mitigation of the model mismatch effect on the estimates. Under some mild regularity conditions, we show that the MT-GQMLE is consistent, asymptotically normal and unbiased. Furthermore, we derive a necessary and sufficient condition for asymptotic efficiency. A data driven procedure for optimal selection of the measure transformation parameters is developed that minimizes the trace of an empirical estimate of the asymptotic mean-squared-error matrix. The MT-GQMLE is applied to linear regression and source localization and numerical comparisons illustrate its robustness and resilience to outliers.

[1]  D. Ruppert.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[2]  Xinmin Yang.  SOME TRACE INEQUALITIES FOR OPERATORS , 1995 .

[3]  H. Vincent Poor,et al.  Complex Elliptically Symmetric Distributions: Survey, New Results and Applications , 2012, IEEE Transactions on Signal Processing.

[4]  Alfred O. Hero,et al.  Robust Multiple Signal Classification via Probability Measure Transformation , 2015, IEEE Transactions on Signal Processing.

[5]  David A. Hsieh,et al.  Modeling Heteroscedasticity in Daily Foreign-Exchange Rates , 1989 .

[6]  S. Kay.  Fundamentals of statistical signal processing: estimation theory , 1993 .

[7]  M. Kendall.  Theoretical Statistics , 1956, Nature.

[8]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[9]  Alfred O. Hero,et al.  Measure transformed canonical correlation analysis with application to financial data , 2012, 2012 IEEE 7th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[10]  Alon Slapak,et al.  Charrelation and Charm: Generic Statistics Incorporating Higher-Order Information , 2012, IEEE Transactions on Signal Processing.

[11]  Jianqing Fan,et al.  Quasi-Maximum Likelihood Estimation of GARCH Models With Heavy-Tailed Likelihoods , 2014 .

[12]  Alon Slapak,et al.  "Weighting for more": Enhancing characteristic-function based ICA with asymptotically optimal weighting , 2011, Signal Process..

[13]  Rory A. Fisher,et al.  Theory of Statistical Estimation , 1925, Mathematical Proceedings of the Cambridge Philosophical Society.

[14]  Tunc Geveci,et al.  Advanced Calculus , 2014, Nature.

[15]  W. Marsden.  I and J , 2012 .

[16]  Alon Slapak,et al.  Charrelation-based estimation of the parameters of non-Gaussian autoregressive processes , 2012, 2012 IEEE Statistical Signal Processing Workshop (SSP).

[17]  R. Fisher,et al.  On the Mathematical Foundations of Theoretical Statistics , 1922 .

[18]  Alfred O. Hero,et al.  On Measure Transformed Canonical Correlation Analysis , 2011, IEEE Transactions on Signal Processing.

[19]  Douglas G. Steigerwald,et al.  Asymptotic Bias for Quasi-Maximum-Likelihood Estimators in Conditional Heteroskedasticity Models , 1997 .

[20]  H. Cramér.  A contribution to the theory of statistical estimation , 1946 .

[21]  T. Bollerslev,et al.  A CONDITIONALLY HETEROSKEDASTIC TIME SERIES MODEL FOR SPECULATIVE PRICES AND RATES OF RETURN , 1987 .

[22]  Panayiotis G. Georgiou,et al.  Maximum likelihood parameter estimation under impulsive conditions, a sub-Gaussian signal approach , 2006, Signal Process..

[23]  J. Wooldridge,et al.  Quasi-maximum likelihood estimation and inference in dynamic models with time-varying covariances , 1992 .

[24]  Inderjit S. Dhillon,et al.  Matrix Nearness Problems with Bregman Divergences , 2007, SIAM J. Matrix Anal. Appl..

[25]  Arie Yeredor,et al.  Blind MIMO identification using the second characteristic function , 2004, IEEE Transactions on Signal Processing.

[26]  Steven Kay,et al.  Maximum Likelihood Estimator Under a Misspecified Model With High Signal-to-Noise Ratio , 2011, IEEE Transactions on Signal Processing.

[27]  L. Scharf,et al.  Statistical Signal Processing of Complex-Valued Data: The Theory of Improper and Noncircular Signals , 2010 .

[28]  P. J. Huber.  The behavior of maximum likelihood estimates under nonstandard conditions , 1967 .

[29]  R. Baillie,et al.  The Message in Daily Exchange Rates , 1989 .

[30]  Alfred O. Hero,et al.  Robust measure transformed music for DOA estimation , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[31]  L. Hansen.  Large Sample Properties of Generalized Method of Moments Estimators , 1982 .

[32]  Bjorn Ottersten,et al.  Exact and Large Sample ML Techniques for Parameter Estimation and Detection in Array Processing , 1993 .

[33]  K. Pearson.  Contributions to the Mathematical Theory of Evolution , 1894 .

[34]  Ing Rj Ser.  Approximation Theorems of Mathematical Statistics , 1980 .

[35]  Halbert White,et al.  Estimation, inference, and specification analysis , 1996 .

[36]  C. Gouriéroux,et al.  PSEUDO MAXIMUM LIKELIHOOD METHODS: THEORY , 1984 .

[37]  A. Enis Çetin,et al.  Robust direction-of-arrival estimation in non-Gaussian noise , 1998, IEEE Trans. Signal Process..

[38]  M. Rangaswamy,et al.  Spherically invariant random processes for modeling non-Gaussian radar clutter , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[39]  Arie Yeredor,et al.  Blind source separation via the second characteristic function , 2000, Signal Process..

[40]  A. Yeredor.  Blind channel estimation using first and second derivatives of the characteristic function , 2002, IEEE Signal Processing Letters.

[41]  Alfred O. Hero,et al.  Measure-transformed quasi maximum likelihood estimation with application to source localization , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[42]  Rudolf Ahlswede,et al.  On minimax estimation in the presence of side information about remote data , 1990 .

[43]  E. Weinstein,et al.  Decentralization of the Gaussian maximum likelihood estimator and its applications to passive array processing , 1981 .

[44]  A. Wald,et al.  On Stochastic Limit and Order Relationships , 1943 .

[45]  Kellen Petersen August.  Real Analysis , 2009 .

[46]  A. Farina,et al.  Vector subspace detection in compound-Gaussian clutter. Part II: performance analysis , 2002 .

[47]  Boaz Porat,et al.  Digital Processing of Random Signals: Theory and Methods , 2008 .

[48]  Harry L. Van Trees,et al.  Optimum Array Processing , 2002 .

[49]  E. L. Lehmann,et al.  Theory of point estimation , 1950 .

[50]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[51]  Visa Koivunen,et al.  Subspace-based direction-of-arrival estimation using nonparametric statistics , 2001, IEEE Trans. Signal Process..

[52]  C. R. Rao,et al.  Information and the Accuracy Attainable in the Estimation of Statistical Parameters , 1992 .

[53]  H. White.  Maximum Likelihood Estimation of Misspecified Models , 1982 .

[54]  Arie Yeredor,et al.  Substituting the cumulants in the super-exponential blind equalization algorithm , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[55]  Douglas B. Williams,et al.  Robust estimation of structured covariance matrices , 1993, IEEE Trans. Signal Process..

[56]  M. Giaquinta,et al.  Mathematical Analysis: An Introduction to Functions of Several Variables , 2004 .

[57]  L. Scharf,et al.  Statistical Signal Processing of Complex-Valued Data: Notation , 2010 .

[58]  Alfred O. Hero,et al.  Relative location estimation in wireless sensor networks , 2003, IEEE Trans. Signal Process..

[59]  A. Yeredor.  MUSIC using Off-Origin Hessians of the Second Characteristic Function , 2006, Fourth IEEE Workshop on Sensor Array and Multichannel Processing, 2006..

[60]  K. Athreya,et al.  Measure Theory and Probability Theory , 2006 .

[61]  M. Braga,et al.  Exploratory Data Analysis , 2018, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[62]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[63]  C. H. Edwards.  Advanced calculus of several variables , 1973 .

[64]  M. Viberg,et al.  Two decades of array signal processing research: the parametric approach , 1996, IEEE Signal Process. Mag..

[65]  E. Conte,et al.  Characterization of radar clutter as a spherically invariant random process , 1987 .

[66]  Marc Moonen,et al.  Gaussian maximum-likelihood channel estimation with short training sequences , 2005, IEEE Transactions on Wireless Communications.

[67]  W. Newey,et al.  Large sample estimation and hypothesis testing , 1986 .