Cram$\acute{\text{e}}$r–Rao Bound for Constrained Parameter Estimation Using Lehmann-Unbiasedness

The constrained Cram$\acute{\text{e}}$r–Rao bound (CCRB) is a lower bound on the mean-squared-error (MSE) of estimators that satisfy some unbiasedness conditions. Although the CCRB unbiasedness conditions are satisfied asymptotically by the constrained maximum likelihood (CML) estimator, in the non-asymptotic region these conditions are usually too strict and the commonly used estimators, such as the CML estimator, do not satisfy them. Therefore, the CCRB may not be a lower bound on the MSE matrix of such estimators. In this paper, we propose a new definition for unbiasedness under constraints, denoted by C-unbiasedness, which is based on using Lehmann-unbiasedness with a weighted MSE (WMSE) risk and taking into account the parametric constraints. In addition to C-unbiasedness, a Cram$\acute{\text{e}}$r–Rao-type bound on the WMSE of C-unbiased estimators, denoted as Lehmann-unbiased CCRB (LU-CCRB), is derived. This bound is a scalar bound that depends on the chosen weighted combination of estimation errors. It is shown that C-unbiasedness is less restrictive than the CCRB unbiasedness conditions. Thus, the set of estimators that satisfy the CCRB unbiasedness conditions is a subset of the set of C-unbiased estimators and the proposed LU-CCRB may be an informative lower bound in cases where the corresponding CCRB is not. In the simulations, we examine linear and nonlinear estimation problems under nonlinear constraints in which the CML estimator is shown to be C-unbiased and the LU-CCRB is an informative lower bound on the WMSE, while the corresponding CCRB on the WMSE is not a lower bound and is not informative in the non-asymptotic region.

[1]  Alfred O. Hero,et al.  Nonlinear Unmixing of Hyperspectral Images: Models and Algorithms , 2013, IEEE Signal Processing Magazine.

[2]  Robert Boorstyn,et al.  Single tone parameter estimation from discrete-time observations , 1974, IEEE Trans. Inf. Theory.

[3]  J. Aitchison,et al.  Maximum-Likelihood Estimation of Parameters Subject to Restraints , 1958 .

[4]  Alfred O. Hero,et al.  On the application of Cramer-Rao type lower bounds for constrained estimation , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[5]  Alle-Jan van der Veen,et al.  Effects of Parametric Constraints on the CRLB in Gain and Phase Estimation Problems , 2006, IEEE Signal Processing Letters.

[6]  M. Crowder On constrained maximum likelihood estimation with non-i.i.d. observations , 1984 .

[7]  Jürgen Pilz,et al.  Minimax linear regression estimation with symmetric parameter restrictions , 1986 .

[8]  H. Cramér A contribution to the theory of statistical estimation , 1946 .

[9]  Yonina C. Eldar,et al.  On the Constrained CramÉr–Rao Bound With a Singular Fisher Information Matrix , 2009, IEEE Signal Processing Letters.

[10]  Joseph Tabrikian,et al.  Bayesian Estimation in the Presence of Deterministic Nuisance Parameters—Part I: Performance Bounds , 2015, IEEE Transactions on Signal Processing.

[11]  Venkatesh Saligrama,et al.  On the Non-Existence of Unbiased Estimators in Constrained Estimation Problems , 2018, IEEE Transactions on Information Theory.

[12]  B. C. Ng,et al.  On the Cramer-Rao bound under parametric constraints , 1998, IEEE Signal Processing Letters.

[13]  Lang Tong,et al.  Joint Frequency and Phasor Estimation Under the KCL Constraint , 2013, IEEE Signal Processing Letters.

[14]  Steven Kay,et al.  Unbiased estimation of the phase of a sinusoid , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[15]  Joseph Tabrikian,et al.  Performance bounds for constrained parameter estimation , 2012, 2012 IEEE 7th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[16]  Joseph Tabrikian,et al.  Non-Bayesian Periodic Cramér-Rao Bound , 2013, IEEE Transactions on Signal Processing.

[17]  Yonina C. Eldar,et al.  The Cramér-Rao Bound for Estimating a Sparse Parameter Vector , 2010, IEEE Transactions on Signal Processing.

[18]  Alfred O. Hero,et al.  Exploring estimator bias-variance tradeoffs using the uniform CR bound , 1996, IEEE Trans. Signal Process..

[19]  G. Styan,et al.  Some further matrix extensions of the Cauchy-Schwarz and Kantorovich inequalities, with some statistical applications , 1996 .

[20]  Fulvio Gini,et al.  The Constrained Misspecified Cramér–Rao Bound , 2016, IEEE Signal Processing Letters.

[21]  C. D. Meyer,et al.  Generalized inverses of linear transformations , 1979 .

[22]  Joseph Tabrikian,et al.  General Classes of Performance Lower Bounds for Parameter Estimation—Part I: Non-Bayesian Bounds for Unbiased Estimators , 2010, IEEE Transactions on Information Theory.

[23]  H. Hendriks A Crame´r-Rao–type lower bound for estimators with values in a manifold , 1991 .

[24]  Alfred O. Hero,et al.  Lower bounds for parametric estimation with constraints , 1990, IEEE Trans. Inf. Theory.

[25]  Bhaskar D. Rao,et al.  Cramer-Rao lower bound for constrained complex parameters , 2004, IEEE Signal Processing Letters.

[26]  T. Moon,et al.  Mathematical Methods and Algorithms for Signal Processing , 1999 .

[27]  Brian M. Sadler,et al.  Bounds on bearing and symbol estimation with side information , 2001, IEEE Trans. Signal Process..

[28]  Nicolas Boumal,et al.  On Intrinsic Cramér-Rao Bounds for Riemannian Submanifolds and Quotient Manifolds , 2013, IEEE Transactions on Signal Processing.

[29]  Brian M. Sadler,et al.  The Constrained CramÉr–Rao Bound From the Perspective of Fitting a Model , 2007, IEEE Signal Processing Letters.

[30]  Zhou Jianxiong,et al.  A New Derivation of Constrained Cramér–Rao Bound Via Norm Minimization , 2011, IEEE Transactions on Signal Processing.

[31]  Eric Chaumette,et al.  Versatility of constrained CRB for system analysis and design , 2014, IEEE Transactions on Aerospace and Electronic Systems.

[32]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[33]  S. D. Silvey,et al.  The Lagrangian Multiplier Test , 1959 .

[34]  Felix Famoye,et al.  Improving Efficiency by Shrinkage , 1999, Technometrics.

[35]  C. R. Rao,et al.  Information and the Accuracy Attainable in the Estimation of Statistical Parameters , 1992 .

[36]  H. V. Trees,et al.  Exploring Estimator BiasVariance Tradeoffs Using the Uniform CR Bound , 2007 .

[37]  Thomas L. Marzetta,et al.  A simple derivation of the constrained multiple parameter Cramer-Rao bound , 1993, IEEE Trans. Signal Process..

[38]  E. Lehmann A General Concept of Unbiasedness , 1951 .

[39]  H. V. Trees,et al.  Covariance, Subspace, and Intrinsic CramrRao Bounds , 2007 .

[40]  Yossef Steinberg,et al.  Extended Ziv-Zakai lower bound for vector parameter estimation , 1997, IEEE Trans. Inf. Theory.

[41]  C. R. Rao,et al.  Pattern recognition based on scale invariant discriminant functions , 1988, Inf. Sci..

[42]  Sheng Chen,et al.  Regularized orthogonal least squares algorithm for constructing radial basis function networks , 1996 .

[43]  T. Moore A Theory of Cramer-Rao Bounds for Constrained Parametric Models , 2010 .

[44]  Joseph Tabrikian,et al.  Limitations of Constrained CRB and an Alternative Bound , 2018, 2018 IEEE Statistical Signal Processing Workshop (SSP).

[45]  Eric Chaumette,et al.  New Results on Deterministic Cramér–Rao Bounds for Real and Complex Parameters , 2012, IEEE Transactions on Signal Processing.

[46]  Yonina C. Eldar Universal Weighted MSE Improvement of the Least-Squares Estimator , 2008, IEEE Transactions on Signal Processing.

[47]  S. Kay Fundamentals of statistical signal processing: estimation theory , 1993 .

[48]  G. Golub,et al.  Quadratically constrained least squares and quadratic problems , 1991 .

[49]  Joseph Tabrikian,et al.  On the limitations of Barankin type bounds for MLE threshold prediction , 2015, Signal Process..

[50]  Anil V. Rao,et al.  Practical Methods for Optimal Control Using Nonlinear Programming , 1987 .

[51]  Michael Elad,et al.  Closed-Form MMSE Estimation for Signal Denoising Under Sparse Representation Modeling Over a Unitary Dictionary , 2010, IEEE Transactions on Signal Processing.

[52]  Ami Wiesel,et al.  Maximum likelihood estimation in linear models with a Gaussian model matrix , 2006, IEEE Signal Processing Letters.

[53]  Gabriela Cohen Freue,et al.  The Constrained Maximum Likelihood Estimation For Parameters Arising From Partially Identified Models , 2016, 1607.08826.

[54]  S.T. Smith,et al.  Covariance, subspace, and intrinsic Crame/spl acute/r-Rao bounds , 2005, IEEE Transactions on Signal Processing.

[55]  Yonina C. Eldar MSE Bounds With Affine Bias Dominating the CramÉr–Rao Bound , 2008, IEEE Transactions on Signal Processing.

[56]  Michael R. Osborne,et al.  Scoring with constraints , 2000, The ANZIAM Journal.

[57]  Lang Tong,et al.  Estimation After Parameter Selection: Performance Analysis and Estimation Methods , 2015, IEEE Transactions on Signal Processing.

[58]  Bin Yang,et al.  MMSE estimation in a linear signal model with ellipsoidal constraints , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[59]  João M. F. Xavier,et al.  Intrinsic variance lower bound (IVLB): an extension of the Cramer-Rao bound to Riemannian manifolds , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[60]  Yonina C. Eldar,et al.  Robust mean-squared error estimation in the presence of model uncertainties , 2005, IEEE Transactions on Signal Processing.

[61]  Eric Chaumette,et al.  A constrained hybrid Cramér-Rao bound for parameter estimation , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[62]  Robinson Piramuthu,et al.  Minimax Emission Computed Tomography using High-Resolution Anatomical Side Information and B-Spline Models , 1999, IEEE Trans. Inf. Theory.

[63]  Joseph Tabrikian,et al.  Cyclic Barankin-Type Bounds for Non-Bayesian Periodic Parameter Estimation , 2014, IEEE Transactions on Signal Processing.

[64]  D. Rajan Probability, Random Variables, and Stochastic Processes , 2017 .

[65]  Petre Stoica,et al.  Linear Regression Constrained to a Ball, , 2001, Digit. Signal Process..

[66]  Brian M. Sadler,et al.  Maximum-Likelihood Estimation, the CramÉr–Rao Bound, and the Method of Scoring With Parameter Constraints , 2008, IEEE Transactions on Signal Processing.