Bounds on achievable convergence rates of parameter estimators via universal coding

Lower bounds on achievable convergence rates of parameter estimators towards the true parameter are derived via universal coding considerations. It is shown that for a parametric class of finite-alphabet information sources, if there exists a universal lossless code whose redundancy decays sufficiently rapidly, then it induces a limitation on the fastest achievable convergence rate of any parameter estimator, at any value of the true parameter, with a possible exception of a vanishingly small subset of parameter values. A specific choice of a universal code yields a slightly different version of this result which extends easily to the continuous case. >

[1]  Jacob Ziv,et al.  Improved Lower Bounds on Signal Parameter Estimation , 1975, IEEE Trans. Inf. Theory.

[2]  D. G. Chapman,et al.  Minimum Variance Estimation Without Regularity Assumptions , 1951 .

[3]  C. R. Rao,et al.  Information and the Accuracy Attainable in the Estimation of Statistical Parameters , 1992 .

[4]  J. Hájek.  Local asymptotic minimax and admissibility in estimation , 1972 .

[5]  G. Darmois,et al.  Sur les limites de la dispersion de certaines estimations , 1945 .

[6]  E. Barankin.  Locally Best Unbiased Estimates , 1949 .

[7]  J. Kiefer.  On Minimum Variance Estimators , 1952 .

[8]  Jorma Rissanen,et al.  Universal coding, information, prediction, and estimation , 1984, IEEE Trans. Inf. Theory.

[9]  M. Fréchet.  Sur l'extension de certaines evaluations statistiques au cas de petits echantillons , 1943 .

[10]  Ehud Weinstein,et al.  A lower bound on the mean-square error in random parameter estimation , 1985, IEEE Trans. Inf. Theory.

[11]  Irwin Guttman,et al.  Bhattacharyya Bounds without Regularity Assumptions , 1952 .

[12]  Sandro Bellini,et al.  Bounds on Error in Signal Parameter Estimation , 1974, IEEE Trans. Commun..

[13]  Ben-Zion Bobrovsky,et al.  A lower bound on the estimation error for certain diffusion processes , 1976, IEEE Trans. Inf. Theory.

[14]  Le Cam,et al.  On some asymptotic properties of maximum likelihood estimates and related Bayes' estimates , 1953 .

[15]  Lee D. Davisson,et al.  Minimax noiseless universal coding for Markov sources , 1983, IEEE Trans. Inf. Theory.

[16]  R. Fisher,et al.  On the Mathematical Foundations of Theoretical Statistics , 1922 .

[17]  Lee D. Davisson,et al.  Universal noiseless coding , 1973, IEEE Trans. Inf. Theory.