Bayesian Estimation in the Presence of Deterministic Nuisance Parameters—Part I: Performance Bounds

How accurately can one estimate a random parameter subject to unknown deterministic nuisance parameters? The hybrid Cramér-Rao bound (HCRB) provides an answer to this question for a restricted class of estimators. The HCRB is the most popular performance bound on the mean-square-error (MSE) for random parameter estimation problems which involve deterministic parameters. The HCRB is useful when one is interested in both the random and the deterministic parameters and in the coupling between their estimation errors. This bound refers to locally weak-sense unbiased estimators with respect to (w.r.t.) the deterministic parameters. However, if these parameters are nuisance, it is unnecessary to restrict their estimation as unbiased. This paper is the first of a two-part study of Bayesian parameter estimation in the presence of deterministic nuisance parameters. It begins with a study on order relations between existing Cramér-Rao (CR)-type bounds of mean-unbiased Bayesian estimators. Then, a new CR-type bound is developed with no assumption of unbiasedness on the nuisance parameters. Alternatively, Lehmann's concept of unbiasedness is employed rather than conventional mean-unbiasedness. It is imposed on a risk that measures the distance between the estimator and the minimum MSE (MMSE) estimator which assumes perfect knowledge of the nuisance parameters. In the succeeding paper, asymptotic performances of some Bayesian estimators with maximum likelihood based estimates for the nuisance parameters are investigated. The proposed risk-unbiased bound (RUB) is proved to be asymptotically achieved by the MMSE estimator with maximum likelihood estimates for the nuisance parameters, while the existing CR-type bounds are not necessarily achievable.

[1]  Joseph Tabrikian,et al.  General Classes of Performance Lower Bounds for Parameter Estimation—Part II: Bayesian Bounds , 2010, IEEE Transactions on Information Theory.

[2]  D. Shinar.  BEN-GURION UNIVERSITY OF THE NEGEV , 2012 .

[3]  Demetrios G. Lainiotis,et al.  Optimal Estimation in the Presence of Unknown Parameters , 1969, IEEE Trans. Syst. Sci. Cybern..

[4]  R. W. Miller,et al.  A modified Cramér-Rao bound and its applications (Corresp.) , 1978, IEEE Trans. Inf. Theory.

[5]  Kristine L. Bell,et al.  A General Class of Lower Bounds in Parameter Estimation , 2007 .

[6]  Joseph Tabrikian,et al.  Bayesian Estimation in the Presence of Deterministic Nuisance Parameters—Part II: Estimation Methods , 2015, IEEE Transactions on Signal Processing.

[7]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[8]  Benjamin Friedlander,et al.  Computation of the exact information matrix of Gaussian time series with stationary random components , 1985, 1985 24th IEEE Conference on Decision and Control.

[9]  Gerard J. Foschini,et al.  Layered space-time architecture for wireless communication in a fading environment when using multi-element antennas , 1996, Bell Labs Technical Journal.

[10]  Hagit Messer,et al.  A Barankin-type lower bound on the estimation error of a hybrid parameter vector , 1997, IEEE Trans. Inf. Theory.

[11]  D. G. Lainiotis,et al.  Optimal adaptive filter realizations for sample stochastic processes with an unknown parameter , 1969 .

[12]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics , 1991 .

[13]  Hagit Messer,et al.  Lower bounds on bearing estimation errors of any number of narrowband far-field sources , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[14]  E. Barankin.  Locally Best Unbiased Estimates , 1949 .

[15]  Joseph Tabrikian,et al.  Cyclic Barankin-Type Bounds for Non-Bayesian Periodic Parameter Estimation , 2014, IEEE Transactions on Signal Processing.

[16]  Harry L. Van Trees,et al.  Detection, Estimation, and Modulation Theory, Part I , 1968 .

[17]  W. F. Trench,et al.  Introduction to Real Analysis: An Educational Approach , 2009 .

[18]  R. Mehra.  On the identification of variances and adaptive Kalman filtering , 1970 .

[19]  Joseph Tabrikian,et al.  Non-Bayesian Periodic Cramér-Rao Bound , 2013, IEEE Transactions on Signal Processing.

[20]  Arie Yeredor,et al.  The joint MAP-ML criterion and its relation to ML and to extended least-squares , 2000, IEEE Trans. Signal Process..

[21]  Peter M. Schultheiss,et al.  Array shape calibration using sources in unknown locations-Part I: Far-field sources , 1987, IEEE Trans. Acoust. Speech Signal Process..

[22]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[23]  D. Spain,et al.  Identification and modelling of discrete stochastic linear systems , 1972, CDC 1972.

[24]  Yushan Xiao,et al.  Linex Unbiasedness in a Prediction Problem , 2000 .

[25]  R. Kashyap.  Maximum likelihood identification of stochastic linear systems , 1970 .

[26]  E. Lehmann.  A General Concept of Unbiasedness , 1951 .

[27]  Marc Moeneclaey,et al.  On the true and the modified Cramer-Rao bounds for the estimation of a scalar parameter in the presence of nuisance parameters , 1998, IEEE Trans. Commun..

[28]  Henry Cox,et al.  On the estimation of state variables and parameters for noisy dynamic systems , 1964 .

[29]  Hagit Messer,et al.  Notes on the Tightness of the Hybrid CramÉr–Rao Lower Bound , 2009, IEEE Transactions on Signal Processing.

[30]  Emil Björnson,et al.  A Framework for Training-Based Estimation in Arbitrarily Correlated Rician MIMO Channels With Rician Disturbance , 2010, IEEE Transactions on Signal Processing.

[31]  K. Steiglitz,et al.  Maximum likelihood estimation of rational transfer function parameters , 1967, IEEE Transactions on Automatic Control.

[32]  G.L. Stuber,et al.  Parameter estimation for OFDM with transmit receive diversity , 2001, IEEE VTS 53rd Vehicular Technology Conference, Spring 2001. Proceedings (Cat. No.01CH37202).

[33]  M. Aoki,et al.  On Certain Convergence Questions in System Identification , 1970 .

[34]  H. Hendriks.  A Crame´r-Rao–type lower bound for estimators with values in a manifold , 1991 .

[35]  M. Levin.  Estimation of a system pulse transfer function in the presence of noise , 1964 .

[36]  Joseph Tabrikian,et al.  Performance bounds for constrained parameter estimation , 2012, 2012 IEEE 7th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[37]  Umberto Mengali,et al.  The modified Cramer-Rao bound in vector parameter estimation , 1998, IEEE Trans. Commun..

[38]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[39]  R. Mehra.  On-line identification of linear dynamic systems with applications to Kalman filtering , 1971 .

[40]  E. L. Lehmann,et al.  Theory of point estimation , 1950 .

[41]  R. Gallager.  Stochastic Processes , 2014 .

[42]  Bingxin Han,et al.  Stochastic System Identification Techniques of State Space Models and its Applications , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[43]  S.K. Wilson,et al.  On channel estimation in OFDM systems , 1995, 1995 IEEE 45th Vehicular Technology Conference. Countdown to the Wireless Twenty-First Century.

[44]  Joseph Tabrikian,et al.  Periodic CRB for non-Bayesian parameter estimation , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[45]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[46]  Joseph Tabrikian,et al.  General Classes of Performance Lower Bounds for Parameter Estimation—Part I: Non-Bayesian Bounds for Unbiased Estimators , 2010, IEEE Transactions on Information Theory.

[47]  Joseph Tabrikian,et al.  Bayesian cramér-rao type bound for risk-unbiased estimation with deterministic nuisance parameters , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[48]  Tapan K. Nayak.  Rao–Cramer Type Inequalities for Mean Squared Error of Prediction , 2002 .

[49]  H. Cramér.  Mathematical methods of statistics , 1947 .

[50]  T. Nayak,et al.  The concept of risk unbiasedness in statistical prediction , 2010 .

[51]  Yannis G. Yatracos,et al.  On prediction and mean squared error , 1992 .

[52]  R. K. Mehra,et al.  Identification of stochastic linear dynamic systems using Kalman filter representation , 1971 .

[53]  Karl Johan Åström,et al.  Numerical Identification of Linear Dynamic Systems from Normal Operating Records , 1965 .

[54]  Y. Bar-Shalom.  Optimal simultaneous state estimation and parameter identification in linear discrete-time systems , 1972 .

[55]  Tapan K. Nayak,et al.  Kshirsagar-Type Lower Bounds for Mean Squared Error of Prediction , 2008 .

[56]  H. Messer.  The Hybrid Cramer-Rao Lower Bound - From Practice to Theory , 2006, Fourth IEEE Workshop on Sensor Array and Multichannel Processing, 2006..

[57]  C. R. Rao,et al.  Information and the Accuracy Attainable in the Estimation of Statistical Parameters , 1992 .

[58]  T. Nayak.  On best unbiased prediction and its relationships to unbiased estimation , 2000 .

[59]  James L. Massey,et al.  Proper complex random processes with applications to information theory , 1993, IEEE Trans. Inf. Theory.