Cramér-Rao Bound for Constrained Parameter Estimation Using Lehmann-Unbiasedness

The constrained Cramér-Rao bound (CCRB) is a lower bound on the mean-squared-error (MSE) of estimators that satisfy some unbiasedness conditions. Although the CCRB unbiasedness conditions are satisfied asymptotically by the constrained maximum likelihood (CML) estimator, in the nonasymptotic region these conditions are usually too strict and the commonly-used estimators, such as the CML estimator, do not satisfy them. Therefore, the CCRB may not be a lower bound on the MSE matrix of such estimators. In this paper, we propose a new definition for unbiasedness under constraints, denoted by C-unbiasedness, which is based on using Lehmann-unbiasedness with a weighted MSE (WMSE) risk and taking into account the parametric constraints. In addition to C-unbiasedness, a CramérRao-type bound on the WMSE of C-unbiased estimators, denoted as Lehmann-unbiased CCRB (LU-CCRB), is derived. This bound is a scalar bound that depends on the chosen weighted combination of estimation errors. It is shown that C-unbiasedness is less restrictive than the CCRB unbiasedness conditions. Thus, the set of estimators that satisfy the CCRB unbiasedness conditions is a subset of the set of C-unbiased estimators and the proposed LUCCRB may be an informative lower bound in cases where the corresponding CCRB is not. In the simulations, we examine linear and nonlinear estimation problems under nonlinear constraints in which the CML estimator is shown to be C-unbiased and the LU-CCRB is an informative lower bound on the WMSE, while the corresponding CCRB on the WMSE is not a lower bound and is not informative in the non-asymptotic region.

[1]  J. Aitchison,et al.  Maximum-Likelihood Estimation of Parameters Subject to Restraints , 1958 .

[2]  S. Kay Fundamentals of statistical signal processing: estimation theory , 1993 .

[3]  Yonina C. Eldar,et al.  The Cramér-Rao Bound for Estimating a Sparse Parameter Vector , 2010, IEEE Transactions on Signal Processing.

[4]  Bin Yang,et al.  MMSE estimation in a linear signal model with ellipsoidal constraints , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[5]  Alfred O. Hero,et al.  Nonlinear Unmixing of Hyperspectral Images: Models and Algorithms , 2013, IEEE Signal Processing Magazine.

[6]  Eric Chaumette,et al.  Versatility of constrained CRB for system analysis and design , 2014, IEEE Transactions on Aerospace and Electronic Systems.

[7]  G. Styan,et al.  Some further matrix extensions of the Cauchy-Schwarz and Kantorovich inequalities, with some statistical applications , 1996 .

[8]  H. V. Trees,et al.  Exploring Estimator BiasVariance Tradeoffs Using the Uniform CR Bound , 2007 .

[9]  Joseph Tabrikian,et al.  General Classes of Performance Lower Bounds for Parameter Estimation—Part I: Non-Bayesian Bounds for Unbiased Estimators , 2010, IEEE Transactions on Information Theory.

[10]  Zhou Jianxiong,et al.  A New Derivation of Constrained Cramér–Rao Bound Via Norm Minimization , 2011, IEEE Transactions on Signal Processing.

[11]  Joseph Tabrikian,et al.  Limitations of Constrained CRB and an Alternative Bound , 2018, 2018 IEEE Statistical Signal Processing Workshop (SSP).

[12]  S. D. Silvey,et al.  The Lagrangian Multiplier Test , 1959 .

[13]  Joseph Tabrikian,et al.  Non-Bayesian Periodic Cramér-Rao Bound , 2013, IEEE Transactions on Signal Processing.

[14]  Joseph Tabrikian,et al.  Performance bounds for constrained parameter estimation , 2012, 2012 IEEE 7th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[15]  Fulvio Gini,et al.  The Constrained Misspecified Cramér–Rao Bound , 2016, IEEE Signal Processing Letters.

[16]  T. Moon,et al.  Mathematical Methods and Algorithms for Signal Processing , 1999 .

[17]  E. L. Lehmann,et al.  Theory of point estimation , 1950 .

[18]  Gabriela Cohen Freue,et al.  The Constrained Maximum Likelihood Estimation For Parameters Arising From Partially Identified Models , 2016, 1607.08826.

[19]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[20]  Robinson Piramuthu,et al.  Minimax Emission Computed Tomography using High-Resolution Anatomical Side Information and B-Spline Models , 1999, IEEE Trans. Inf. Theory.

[21]  M. Crowder On constrained maximum likelihood estimation with non-i.i.d. observations , 1984 .

[22]  Lang Tong,et al.  Estimation After Parameter Selection: Performance Analysis and Estimation Methods , 2015, IEEE Transactions on Signal Processing.

[23]  Thomas L. Marzetta,et al.  A simple derivation of the constrained multiple parameter Cramer-Rao bound , 1993, IEEE Trans. Signal Process..

[24]  C. R. Rao,et al.  Pattern recognition based on scale invariant discriminant functions , 1988, Inf. Sci..

[25]  Michael R. Osborne,et al.  Scoring with constraints , 2000, The ANZIAM Journal.

[26]  Steven Kay,et al.  Unbiased estimation of the phase of a sinusoid , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[27]  Michael Elad,et al.  Closed-Form MMSE Estimation for Signal Denoising Under Sparse Representation Modeling Over a Unitary Dictionary , 2010, IEEE Transactions on Signal Processing.

[28]  Alfred O. Hero,et al.  Lower bounds for parametric estimation with constraints , 1990, IEEE Trans. Inf. Theory.

[29]  Joseph Tabrikian,et al.  Cyclic Barankin-Type Bounds for Non-Bayesian Periodic Parameter Estimation , 2014, IEEE Transactions on Signal Processing.

[30]  Eric Chaumette,et al.  A constrained hybrid Cramér-Rao bound for parameter estimation , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[31]  Jürgen Pilz,et al.  Minimax linear regression estimation with symmetric parameter restrictions , 1986 .

[32]  Yonina C. Eldar,et al.  On the Constrained CramÉr–Rao Bound With a Singular Fisher Information Matrix , 2009, IEEE Signal Processing Letters.

[33]  Lang Tong,et al.  Joint Frequency and Phasor Estimation Under the KCL Constraint , 2013, IEEE Signal Processing Letters.

[34]  H. Hendriks A Crame´r-Rao–type lower bound for estimators with values in a manifold , 1991 .

[35]  Sheng Chen,et al.  Regularized orthogonal least squares algorithm for constructing radial basis function networks , 1996 .

[36]  T. Moore A Theory of Cramer-Rao Bounds for Constrained Parametric Models , 2010 .

[37]  Petre Stoica,et al.  Linear Regression Constrained to a Ball, , 2001, Digit. Signal Process..

[38]  João M. F. Xavier,et al.  Intrinsic variance lower bound (IVLB): an extension of the Cramer-Rao bound to Riemannian manifolds , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[39]  Bhaskar D. Rao,et al.  Cramer-Rao lower bound for constrained complex parameters , 2004, IEEE Signal Processing Letters.

[40]  Alle-Jan van der Veen,et al.  Effects of Parametric Constraints on the CRLB in Gain and Phase Estimation Problems , 2006, IEEE Signal Processing Letters.

[41]  Nicolas Boumal,et al.  On Intrinsic Cramér-Rao Bounds for Riemannian Submanifolds and Quotient Manifolds , 2013, IEEE Transactions on Signal Processing.

[42]  Yonina C. Eldar MSE Bounds With Affine Bias Dominating the CramÉr–Rao Bound , 2008, IEEE Transactions on Signal Processing.

[43]  Felix Famoye,et al.  Improving Efficiency by Shrinkage , 1999, Technometrics.

[44]  Yonina C. Eldar Universal Weighted MSE Improvement of the Least-Squares Estimator , 2008, IEEE Transactions on Signal Processing.

[45]  G. Golub,et al.  Quadratically constrained least squares and quadratic problems , 1991 .

[46]  Alfred O. Hero,et al.  On the application of Cramer-Rao type lower bounds for constrained estimation , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[47]  C. R. Rao,et al.  Information and the Accuracy Attainable in the Estimation of Statistical Parameters , 1992 .

[48]  Robert Boorstyn,et al.  Single tone parameter estimation from discrete-time observations , 1974, IEEE Trans. Inf. Theory.

[49]  Eric Chaumette,et al.  New Results on Deterministic Cramér–Rao Bounds for Real and Complex Parameters , 2012, IEEE Transactions on Signal Processing.

[50]  H. V. Trees,et al.  Covariance, Subspace, and Intrinsic CramrRao Bounds , 2007 .

[51]  Joseph Tabrikian,et al.  On the limitations of Barankin type bounds for MLE threshold prediction , 2015, Signal Process..

[52]  Yossef Steinberg,et al.  Extended Ziv-Zakai lower bound for vector parameter estimation , 1997, IEEE Trans. Inf. Theory.

[53]  B. C. Ng,et al.  On the Cramer-Rao bound under parametric constraints , 1998, IEEE Signal Processing Letters.

[54]  Venkatesh Saligrama,et al.  On the Non-Existence of Unbiased Estimators in Constrained Estimation Problems , 2018, IEEE Transactions on Information Theory.

[55]  Brian M. Sadler,et al.  Bounds on bearing and symbol estimation with side information , 2001, IEEE Trans. Signal Process..

[56]  E. Lehmann A General Concept of Unbiasedness , 1951 .

[57]  Yonina C. Eldar,et al.  Robust mean-squared error estimation in the presence of model uncertainties , 2005, IEEE Transactions on Signal Processing.

[58]  C. D. Meyer,et al.  Generalized inverses of linear transformations , 1979 .

[59]  Brian M. Sadler,et al.  The Constrained CramÉr–Rao Bound From the Perspective of Fitting a Model , 2007, IEEE Signal Processing Letters.

[60]  Anil V. Rao,et al.  Practical Methods for Optimal Control Using Nonlinear Programming , 1987 .

[61]  Brian M. Sadler,et al.  Maximum-Likelihood Estimation, the CramÉr–Rao Bound, and the Method of Scoring With Parameter Constraints , 2008, IEEE Transactions on Signal Processing.

[62]  Ami Wiesel,et al.  Maximum likelihood estimation in linear models with a Gaussian model matrix , 2006, IEEE Signal Processing Letters.

[63]  Joseph Tabrikian,et al.  Bayesian Estimation in the Presence of Deterministic Nuisance Parameters—Part I: Performance Bounds , 2015, IEEE Transactions on Signal Processing.

[64]  H. Cramér A contribution to the theory of statistical estimation , 1946 .