On Optimum Parameter Modulation–Estimation From a Large Deviations Perspective

We consider the problem of jointly optimum modulation and estimation of a real-valued random parameter, conveyed over an additive white Gaussian noise (AWGN) channel, where the performance metric is the large deviations behavior of the estimator, namely the exponential decay rate (as a function of the observation time) of the probability that the estimation error would exceed a certain threshold. Our basic result is in providing an exact characterization of the fastest achievable exponential decay rate, among all possible modulator-estimator (transmitter-receiver) pairs, where the modulator is limited only in the signal power, but not in bandwidth. This exponential rate turns out to be given by the reliability function of the AWGN channel. We also discuss several ways to achieve this optimum performance, and one of them is based on quantization of the parameter, followed by optimum channel coding and modulation, which gives rise to a separation-based transmitter, if one views this setting from the perspective of joint source-channel coding. This is in spite of the fact that, in general, when error exponents are considered, the source-channel separation theorem does not hold true. We also discuss several observations, modifications, and extensions of this result in several directions, including other channels, and the case of multidimensional parameter vectors. One of our findings concerning the latter is that there is an abrupt threshold effect in the dimensionality of the parameter vector: below a certain critical dimension, the probability of excess estimation error may still decay exponentially, but beyond this value, it must converge to unity.

[1]  J. Ziv,et al.  A Generalization of the Rate-Distortion Theory and Applications , 1975 .

[2]  Neri Merhav,et al.  Universal composite hypothesis testing: A competitive minimax approach , 2002, IEEE Trans. Inf. Theory.

[3]  G. Longo Source Coding Theory , 1970 .

[4]  Jacob Ziv,et al.  On the threshold effect in radar range estimation (Corresp.) , 1969, IEEE Trans. Inf. Theory.

[5]  Katalin Marton,et al.  Error exponent for source coding with a fidelity criterion , 1974, IEEE Trans. Inf. Theory.

[6]  Imre Csiszár On the error exponent of source-channel transmission with a distortion threshold , 1982, IEEE Trans. Inf. Theory.

[7]  Yonina C. Eldar,et al.  A Lower Bound on the Bayesian MSE Based on the Optimal Bias Function , 2008, IEEE Transactions on Information Theory.

[8]  Neri Merhav,et al.  Threshold Effects in Parameter Estimation as Phase Transitions in Statistical Mechanics , 2010, IEEE Transactions on Information Theory.

[9]  A. Wyner,et al.  On communication of analog data from a bounded source space , 1969 .

[10]  R. Gallager ENERGY LIMITED CHANNELS : CODING , MULTIACCESS , AND SPREAD SPECTRUM * * ABSTRACT , 1987 .

[11]  Aaron D. Wyner,et al.  Capacity and error exponent for the direct detection photon channel-Part I , 1988, IEEE Trans. Inf. Theory.

[12]  Neri Merhav Data-Processing Inequalities Based on a Certain Structured Class of Information Measures With Application to Estimation Theory , 2012, IEEE Transactions on Information Theory.

[13]  Rodger E. Ziemer,et al.  Principles of Communication , 2001 .

[14]  Harry L. Van Trees,et al.  Detection, Estimation, and Modulation Theory, Part I , 1968 .

[15]  R. Gallager Power Limited Channels: Coding, Multiaccess, and Spread Spectrum , 2002 .

[16]  E. L. Lehmann,et al.  Theory of point estimation , 1950 .

[17]  Jacob Ziv,et al.  On functionals satisfying a data-processing theorem , 1973, IEEE Trans. Inf. Theory.

[18]  Yossef Steinberg,et al.  Extended Ziv-Zakai lower bound for vector parameter estimation , 1997, IEEE Trans. Inf. Theory.

[19]  B.D. Steinberg,et al.  Principles of communication , 1978, Proceedings of the IEEE.

[20]  C. Tellambura,et al.  Derivation of Craig's formula for Gaussian probability function , 1999 .

[21]  I. M. Jacobs,et al.  Principles of Communication Engineering , 1965 .

[22]  D. A. Bell,et al.  Information Theory and Reliable Communication , 1969 .

[23]  Tsachy Weissman,et al.  Joint source-channel coding of one random variable over the Poisson channel , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[24]  Jacob Ziv,et al.  Improved Lower Bounds on Signal Parameter Estimation , 1975, IEEE Trans. Inf. Theory.

[25]  Seymour Sherman,et al.  Non-mean-square error criteria , 1958, IRE Trans. Inf. Theory.

[26]  V.W.S. Chan,et al.  Principles of Digital Communication and Coding , 1979 .

[27]  Aaron D. Wyner,et al.  Capacity and error-exponent for the direct detection photon channel-Part II , 1988, IEEE Trans. Inf. Theory.

[28]  Jacob Ziv,et al.  Some lower bounds on signal parameter estimation , 1969, IEEE Trans. Inf. Theory.

[29]  A. Kester,et al.  Large Deviations of Estimators , 1986 .