Parameter Bounds on Estimation Accuracy Under Model Misspecification

When the assumed data distribution differs from the true distribution, the model is said to be misspecified or mismatched. Model misspecification at some level is an inevitability of engineering practice. While Huber's celebrated work assesses maximum-likelihood (ML) performance under misspecification, no simple theory for bounding parameter estimation exists. The class of parameter bounds emerging from the covariance inequality, or equivalently the minimum norm theorem is revisited. The expectation operator is well-known to form an inner product space. Flexibility in the choice of expectation integrand and measure for integration exists, however, to establish a class of parameter bounds under a general form of model misspecification, i.e., distribution mismatch. The Cramér-Rao bound (CRB) primarily, and secondarily the Barankin/Hammersley-Chapman-Robbins, Bhattacharyya, and Bobrovsky-Mayer-Wolf-Zakai bounds under misspecification are considered. Huber's sandwich covariance is readily established as a special case of the misspecified CRB subject to ML constraints, and generalizations of the Slepian and Bangs formulae under misspecification are obtained.

[1]  Claudia Biermann,et al.  Mathematical Methods Of Statistics , 2016 .

[2]  A. Bos Complex gradient and Hessian , 1994 .

[3]  Philippe Forster,et al.  On lower bounds for deterministic parameter estimation , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[4]  E. Weinstein,et al.  Lower bounds on the mean square estimation error , 1985, Proceedings of the IEEE.

[5]  Adriaan van den Bos,et al.  A Cramer-Rao lower bound for complex parameters , 1994, IEEE Trans. Signal Process..

[6]  E. Barankin Locally Best Unbiased Estimates , 1949 .

[7]  E. L. Lehmann,et al.  Theory of point estimation , 1950 .

[8]  H. V. Trees,et al.  Bayesian Bounds for Parameter Estimation and Nonlinear Filtering/Tracking , 2007 .

[9]  Thomas M. Cover,et al.  Elements of information theory (2. ed.) , 2006 .

[10]  H. White Maximum Likelihood Estimation of Misspecified Models , 1982 .

[11]  Christ D. Richmond,et al.  Mean-squared error and threshold SNR prediction of maximum-likelihood signal parameter estimation with estimated colored noise covariances , 2006, IEEE Transactions on Information Theory.

[12]  Thomas L. Marzetta,et al.  Detection, Estimation, and Modulation Theory , 1976 .

[13]  Phillipp Meister,et al.  Statistical Signal Processing Detection Estimation And Time Series Analysis , 2016 .

[14]  Ehud Weinstein,et al.  A general class of lower bounds in parameter estimation , 1988, IEEE Trans. Inf. Theory.

[15]  Philippe Forster,et al.  A Fresh Look at the Bayesian Bounds of the Weiss-Weinstein Family , 2008, IEEE Transactions on Signal Processing.

[16]  Louis L. Scharf,et al.  Properties of quadratic covariance bounds , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[17]  David S. Slepian,et al.  Estimation of signal parameters in the presence of noise , 1954, Trans. IRE Prof. Group Inf. Theory.

[18]  Jonathan S. Abel,et al.  A bound on mean-square-estimate error , 1993, IEEE Trans. Inf. Theory.

[19]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[20]  H. Akaike A new look at the statistical model identification , 1974 .

[21]  R. Remmert,et al.  Theory of Complex Functions , 1990 .

[22]  I. Miller Probability, Random Variables, and Stochastic Processes , 1966 .

[23]  Wen Xu,et al.  A bound on mean-square estimation error with background parameter mismatch , 2004, IEEE Transactions on Information Theory.

[24]  Thomas L. Marzetta Computing the Barankin bound, by solving an unconstrained quadratic optimization problem , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[25]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[26]  P. J. Huber The behavior of maximum likelihood estimates under nonstandard conditions , 1967 .

[27]  B. Hofmann-Wellenhof,et al.  Introduction to spectral analysis , 1986 .

[28]  M. Zakai,et al.  Some Classes of Global Cramer-Rao Bounds , 1987 .

[29]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[30]  A. Wald Note on the Consistency of the Maximum Likelihood Estimate , 1949 .

[31]  C. R. Rao,et al.  Information and the Accuracy Attainable in the Estimation of Statistical Parameters , 1992 .

[32]  B. A. D. H. Brandwood A complex gradient operator and its applica-tion in adaptive array theory , 1983 .