Exponential Error Bounds on Parameter Modulation–Estimation for Discrete Memoryless Channels

We consider the problem of modulation and estimation of a random parameter U to be conveyed across a discrete memoryless channel. Upper and lower bounds are derived for the best achievable exponential decay rate of a general moment of the estimation error, E|Û-U|ρ, ρ ≥ 0, when both the modulator and the estimator are subjected to optimization. These exponential error bounds turn out to be intimately related to error exponents of channel coding and to channel capacity. While in general, there is some gap between the upper and the lower bounds, they asymptotically coincide both for very small and for very large values of the moment power ρ. This means that our achievability scheme, which is based on simple quantization of U followed by channel coding, is nearly optimum in both limits. Some additional properties of the bounds are discussed and demonstrated, and finally, an extension to the case of a multidimensional parameter vector is outlined, with the principal conclusion that our upper and lower bounds asymptotically coincide also for a high dimensionality.

[1]  H. V. Trees,et al.  Bayesian Bounds for Parameter Estimation and Nonlinear Filtering/Tracking , 2007 .

[2]  Lizhong Zheng,et al.  Upper bounds to error probability with feedback , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[3]  J. Ziv,et al.  A Generalization of the Rate-Distortion Theory and Applications , 1975 .

[4]  D. A. Bell,et al.  Information Theory and Reliable Communication , 1969 .

[5]  Tsachy Weissman,et al.  Joint source-channel coding of one random variable over the Poisson channel , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[6]  Jacob Ziv,et al.  Improved Lower Bounds on Signal Parameter Estimation , 1975, IEEE Trans. Inf. Theory.

[7]  V.W.S. Chan,et al.  Principles of Digital Communication and Coding , 1979 .

[8]  Brian L. Hughes,et al.  A new universal random coding bound for the multiple-access channel , 1996, IEEE Trans. Inf. Theory.

[9]  Neri Merhav Universal Decoding for Arbitrary Channels Relative to a Given Class of Decoding Metrics , 2013, IEEE Transactions on Information Theory.

[10]  Yonina C. Eldar,et al.  A Lower Bound on the Bayesian MSE Based on the Optimal Bias Function , 2008, IEEE Transactions on Information Theory.

[11]  I. M. Jacobs,et al.  Principles of Communication Engineering , 1965 .

[12]  Hans-Martin Wallmeier,et al.  Random coding bound and codes produced by permutations for the multiple-access channel , 1985, IEEE Trans. Inf. Theory.

[13]  N. Sloane,et al.  Lower Bounds to Error Probability for Coding on Discrete Memoryless Channels. I , 1993 .

[14]  Ram Zamir,et al.  Bounds for joint source-channel coding at high SNR , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[15]  Neri Merhav On Optimum Parameter Modulation–Estimation From a Large Deviations Perspective , 2012, IEEE Transactions on Information Theory.

[16]  Jacob Ziv,et al.  On functionals satisfying a data-processing theorem , 1973, IEEE Trans. Inf. Theory.

[17]  Yossef Steinberg,et al.  Extended Ziv-Zakai lower bound for vector parameter estimation , 1997, IEEE Trans. Inf. Theory.

[18]  B.D. Steinberg,et al.  Principles of communication , 1978, Proceedings of the IEEE.

[19]  Ram Zamir,et al.  A Ziv-Zakai-Rényi lower bound on distortion at high resolution , 2008, 2008 IEEE Information Theory Workshop.

[20]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[21]  Neri Merhav Data-Processing Inequalities Based on a Certain Structured Class of Information Measures With Application to Estimation Theory , 2012, IEEE Transactions on Information Theory.

[22]  Jacob Ziv,et al.  Some lower bounds on signal parameter estimation , 1969, IEEE Trans. Inf. Theory.

[23]  Elwyn R. Berlekamp,et al.  Lower Bounds to Error Probability for Coding on Discrete Memoryless Channels. II , 1967, Inf. Control..