Computing Constrained Cramér-Rao Bounds

We revisit the problem of computing submatrices of the Cramér-Rao bound (CRB), which lower bounds the variance of any unbiased estimator of a vector parameter \mbi θ. We explore iterative methods that avoid direct inversion of the Fisher information matrix, which can be computationally expensive when the dimension of \mbi θ is large. The computation of the bound is related to the quadratic matrix program, where there are highly efficient methods for solving it. We present several methods, and show that algorithms in prior work are special instances of existing optimization algorithms. Some of these methods converge to the bound monotonically, but in particular, algorithms converging nonmonotonically are much faster. We then extend the work to encompass the computation of the CRB when the Fisher information matrix is singular and when the parameter \mbi θ is subject to constraints. As an application, we consider the design of a data streaming algorithm for network measurement.

[1]  Donald F. Towsley,et al.  A resource-minimalist flow size histogram estimator , 2008, IMC '08.

[2]  B. C. Ng,et al.  On the Cramer-Rao bound under parametric constraints , 1998, IEEE Signal Processing Letters.

[3]  A. Hero,et al.  A recursive algorithm for computing Cramer-Rao- type bounds on estimator covariance , 1994, IEEE Trans. Inf. Theory.

[4]  Amir Beck,et al.  Quadratic Matrix Programming , 2006, SIAM J. Optim..

[5]  D. Hunter,et al.  A Tutorial on MM Algorithms , 2004 .

[6]  Steven Kay,et al.  Fundamentals Of Statistical Signal Processing , 2001 .

[7]  G. M. Ostrovsky,et al.  Linearly constrained optimization , 1989, Computing.

[8]  L. Brown,et al.  Nonexistence of Informative Unbiased Estimators in Singular Problems , 1993 .

[9]  S. Kay Fundamentals of statistical signal processing: estimation theory , 1993 .

[10]  Alfred O. Hero,et al.  Lower bounds for parametric estimation with constraints , 1990, IEEE Trans. Inf. Theory.

[11]  Christoph F. Mecklenbräuker,et al.  Multidimensional Rank Reduction Estimator for Parametric MIMO Channel Models , 2004, EURASIP J. Adv. Signal Process..

[12]  W. Press,et al.  Numerical Recipes: The Art of Scientific Computing , 1987 .

[13]  David S. Watkins,et al.  Fundamentals of matrix computations , 1991 .

[14]  Abhishek Kumar,et al.  Data streaming algorithms for efficient and accurate estimation of flow size distribution , 2004, SIGMETRICS '04/Performance '04.

[15]  O. Strand Theory and methods related to the singular-function expansion and Landweber's iteration for integral equations of the first kind , 1974 .

[16]  Nikos D. Sidiropoulos,et al.  Cramer-Rao lower bounds for low-rank decomposition of multidimensional arrays , 2001, IEEE Trans. Signal Process..

[17]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[18]  David S. Watkins,et al.  Fundamentals of Matrix Computations: Watkins/Fundamentals of Matrix Computations , 2005 .

[19]  Alfred O. Hero,et al.  Recursive algorithms for computing the Cramer-Rao bound , 1997, IEEE Trans. Signal Process..

[20]  D. Harville Matrix Algebra From a Statistician's Perspective , 1998 .

[21]  Darryl Veitch,et al.  Fisher Information in Flow Size Distribution Estimation , 2011, IEEE Transactions on Information Theory.

[22]  R. Boyer Decoupled root-MUSIC algorithm for Multidimensional Harmonic retrieval , 2008, 2008 IEEE 9th Workshop on Signal Processing Advances in Wireless Communications.

[23]  Darryl Veitch,et al.  Sampling vs sketching: An information theoretic comparison , 2011, 2011 Proceedings IEEE INFOCOM.

[24]  Alfred O. Hero,et al.  Exploring estimator bias-variance tradeoffs using the uniform CR bound , 1996, IEEE Trans. Signal Process..

[25]  Thomas L. Marzetta,et al.  Parameter estimation problems with singular information matrices , 2001, IEEE Trans. Signal Process..

[26]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.