Optimal biased estimation using Lehmann-unbiasedness

This paper deals with non-Bayesian parameter estimation under the mean-squared-error (MSE), which is a topic of great interest in various engineering fields. Although the unbiasedness condition is commonly used in non-Bayesian MSE estimation, in many cases biased estimation may result in better performance. However, no method for determining the optimal bias function in general cases is available. We propose a new approach for uniform minimum MSE biased estimation, where the optimal bias is chosen in accordance with Lehmann-unbiasedness definition. The proposed approach is based on modifying the MSE risk by its multiplication with a weighting function of the unknown parameter, g2. Under this modified risk, Lehmann's definition of unbiasedness provides a condition referred to as g-unbiasedness. By using the g-unbiasedness, we derive a novel Cramér-Rao-type lower bound on the MSE of locally g-unbiased estimators. In addition, we show that if there exists an estimator that achieves the new bound, then it is produced by the penalized maximum likelihood estimator with a penalty function log g. Simulations show that the proposed approach can lead to non-trivial estimators with lower MSE than existing mean-unbiased estimators.

[1]  Steven Kay,et al.  Unbiased estimation of the phase of a sinusoid , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[2]  Joseph Tabrikian,et al.  Non-Bayesian Periodic Cramér-Rao Bound , 2013, IEEE Transactions on Signal Processing.

[3]  Alfred O. Hero,et al.  Exploring estimator bias-variance tradeoffs using the uniform CR bound , 1996, IEEE Trans. Signal Process..

[4]  Eric Chaumette,et al.  New Results on Deterministic Cramér–Rao Bounds for Real and Complex Parameters , 2012, IEEE Transactions on Signal Processing.

[5]  C. Stein,et al.  Estimation with Quadratic Loss , 1992 .

[6]  Joseph Tabrikian,et al.  New observations on efficiency of variance estimation of white Gaussian signal with unknown mean , 2016, 2016 IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM).

[7]  Alfred O. Hero,et al.  Ieee Transactions on Image Processing: to Appear Penalized Maximum-likelihood Image Reconstruction Using Space-alternating Generalized Em Algorithms , 2022 .

[8]  Joseph Tabrikian,et al.  Cyclic Barankin-Type Bounds for Non-Bayesian Periodic Parameter Estimation , 2014, IEEE Transactions on Signal Processing.

[9]  S. Kay Fundamentals of statistical signal processing: estimation theory , 1993 .

[10]  Alfred O. Hero,et al.  Robust Shrinkage Estimation of High-Dimensional Covariance Matrices , 2010, IEEE Transactions on Signal Processing.

[11]  Yonina C. Eldar,et al.  Rethinking biased estimation [Lecture Notes] , 2008, IEEE Signal Processing Magazine.

[12]  Eric Chaumette,et al.  A New Barankin Bound Approximation for the Prediction of the Threshold Region Performance of Maximum Likelihood Estimators , 2008, IEEE Transactions on Signal Processing.

[13]  Joseph Tabrikian,et al.  On the limitations of Barankin type bounds for MLE threshold prediction , 2015, Signal Process..

[14]  Yonina C. Eldar,et al.  Robust mean-squared error estimation in the presence of model uncertainties , 2005, IEEE Transactions on Signal Processing.

[15]  R. A. Gaskins,et al.  Nonparametric roughness penalties for probability densities , 2022 .

[16]  B. Blight Some General Results on Reduced Mean Square Error Estimation , 1971 .

[17]  Peter Jancovic,et al.  Acoustic Recognition of Multiple Bird Species Based on Penalized Maximum Likelihood , 2015, IEEE Signal Processing Letters.

[18]  Dengyuan Xu,et al.  Weighted Linear Least Square Localization Algorithms for Received Signal Strength , 2013, Wirel. Pers. Commun..

[19]  T. Kariya Equivariant Estimation in a Model with an Ancillary Statistic , 1989 .

[20]  Zhi Ding,et al.  Distance Estimation From Received Signal Strength Under Log-Normal Shadowing: Bias and Variance , 2008, IEEE Signal Processing Letters.

[21]  H. Cramér A contribution to the theory of statistical estimation , 1946 .

[22]  Joseph Tabrikian,et al.  Cyclic Cramér-Rao-type bounds for periodic parameter estimation , 2016, 2016 19th International Conference on Information Fusion (FUSION).

[23]  Yonina C. Eldar Rethinking Biased Estimation: Improving Maximum Likelihood and the Cramér-Rao Bound , 2008, Found. Trends Signal Process..

[24]  Joseph Tabrikian,et al.  General Classes of Performance Lower Bounds for Parameter Estimation—Part I: Non-Bayesian Bounds for Unbiased Estimators , 2010, IEEE Transactions on Information Theory.

[25]  Joseph Tabrikian,et al.  Performance bounds for constrained parameter estimation , 2012, 2012 IEEE 7th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[26]  H. V. Trees,et al.  Exploring Estimator BiasVariance Tradeoffs Using the Uniform CR Bound , 2007 .

[27]  E. Lehmann A General Concept of Unbiasedness , 1951 .

[28]  Joseph Tabrikian,et al.  Uniformly Best Biased Estimators in Non-Bayesian Parameter Estimation , 2011, IEEE Transactions on Information Theory.

[29]  Yonina C. Eldar MSE Bounds With Affine Bias Dominating the CramÉr–Rao Bound , 2008, IEEE Transactions on Signal Processing.

[30]  Joseph Tabrikian,et al.  A risk-unbiased bound for information fusion with nuisance parameters , 2016, 2016 19th International Conference on Information Fusion (FUSION).

[31]  C. R. Rao,et al.  Information and the Accuracy Attainable in the Estimation of Statistical Parameters , 1992 .

[32]  Petre Stoica,et al.  On biased estimators and the unbiased Cramér-Rao lower bound , 1990, Signal Process..

[33]  Jonathan S. Abel,et al.  A bound on mean-square-estimate error , 1993, IEEE Trans. Inf. Theory.

[34]  Jeffrey A. Fessler Mean and variance of implicitly defined biased estimators (such as penalized maximum likelihood): applications to tomography , 1996, IEEE Trans. Image Process..

[35]  Yonina C. Eldar Uniformly Improving the CramÉr-Rao Bound and Maximum-Likelihood Estimation , 2006, IEEE Transactions on Signal Processing.

[36]  Yonina C. Eldar Minimum variance in biased estimation: bounds and asymptotically optimal estimators , 2004, IEEE Transactions on Signal Processing.

[37]  Joseph Tabrikian,et al.  Bayesian Estimation in the Presence of Deterministic Nuisance Parameters—Part I: Performance Bounds , 2015, IEEE Transactions on Signal Processing.

[38]  Lang Tong,et al.  Estimation After Parameter Selection: Performance Analysis and Estimation Methods , 2015, IEEE Transactions on Signal Processing.

[39]  Joseph Tabrikian,et al.  Mean-cyclic-error lower bounds via integral transform of likelihood-ratio function , 2016, 2016 IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM).

[40]  Tirza Routtenberg Two-stage estimation after parameter selection , 2016, 2016 IEEE Statistical Signal Processing Workshop (SSP).

[41]  Joseph Tabrikian,et al.  A risk-unbiased approach to a new Cramér-Rao bound , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[42]  E. Barankin Locally Best Unbiased Estimates , 1949 .