Asymptotic Design of Quantizers for Decentralized MMSE Estimation

Conceptual and practical encoding/decoding, aimed at accurately reproducing remotely collected observations, has been heavily investigated since the pioneering works by Shannon about source coding. However, when the goal is not to reproduce the observables, but making inference about an embedded parameter and the scenario consists of many unconnected remote nodes, the landscape is less certain. We consider a multiterminal system designed for efficiently estimating a random parameter according to the minimum mean square error (MMSE) criterion. The analysis is limited to scalar quantizers followed by a joint entropy encoder, and it is performed in the high-resolution regime where the problem can be more easily mathematically tackled. Focus is made on the peculiarities deriving from the estimation task, as opposed to that of reconstruction, as well as on the multiterminal, as opposite to centralized, character of the inference. The general form of the optimal nonuniform quantizer is derived and examples are given.

[1]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[2]  Herbert Gish,et al.  Asymptotically efficient quantizing , 1968, IEEE Trans. Inf. Theory.

[3]  Toby Berger Minimum entropy quantizers and permutation codes , 1982, IEEE Trans. Inf. Theory.

[4]  Jacob Ziv,et al.  On universal quantization , 1985, IEEE Trans. Inf. Theory.

[5]  H. Poor,et al.  Fine quantization in signal detection and estimation , 1988, IEEE Trans. Inf. Theory.

[6]  Harry L. Van Trees,et al.  Detection, Estimation, and Modulation Theory, Part I , 1968 .

[7]  Hirosuke Yamamoto,et al.  Source Coding Theory for Multiterminal Communication Systems with a Remote Source , 1980 .

[8]  Toby Berger Optimum quantizers and permutation codes , 1972, IEEE Trans. Inf. Theory.

[9]  A. Gualtierotti H. L. Van Trees, Detection, Estimation, and Modulation Theory, , 1976 .

[10]  Toby Berger,et al.  Multiterminal Source Coding with High Resolution , 1999, IEEE Trans. Inf. Theory.

[11]  Robert M. Gray,et al.  Encoding of correlated observations , 1987, IEEE Trans. Inf. Theory.

[12]  Terrence L. Fine,et al.  Optimum mean-square quantization of a noisy input (Corresp.) , 1965, IEEE Trans. Inf. Theory.

[13]  Zhen Zhang,et al.  On the CEO problem , 1994, Proceedings of 1994 IEEE International Symposium on Information Theory.

[14]  John N. Tsitsiklis,et al.  Decentralized detection by a large number of sensors , 1988, Math. Control. Signals Syst..

[15]  J. G. Gander,et al.  An introduction to signal detection and estimation , 1990 .

[16]  Harish Viswanathan,et al.  On the whiteness of high-resolution quantization errors , 2000, IEEE Trans. Inf. Theory.

[17]  Boris Tsybakov,et al.  Information transmission with additional noise , 1962, IRE Trans. Inf. Theory.

[18]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[19]  M. Melamed Detection , 2021, SETI: Astronomy as a Contact Sport.

[20]  S. Schor STATISTICS: AN INTRODUCTION. , 1965, The Journal of trauma.

[21]  Ehud Weinstein,et al.  A general class of lower bounds in parameter estimation , 1988, IEEE Trans. Inf. Theory.

[22]  Jack K. Wolf,et al.  Noiseless coding of correlated information sources , 1973, IEEE Trans. Inf. Theory.

[23]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[24]  Toby Berger,et al.  An upper bound on the sum-rate distortion function and its corresponding rate allocation schemes for the CEO problem , 2004, IEEE Journal on Selected Areas in Communications.

[25]  Jack K. Wolf,et al.  Transmission of noisy information to a noisy receiver with minimum distortion , 1970, IEEE Trans. Inf. Theory.

[26]  Shun-ichi Amari,et al.  Statistical Inference Under Multiterminal Data Compression , 1998, IEEE Trans. Inf. Theory.

[27]  Yasutada Oohama,et al.  The Rate-Distortion Function for the Quadratic Gaussian CEO Problem , 1998, IEEE Trans. Inf. Theory.

[28]  B. Girod,et al.  Wyner-Ziv quantization and transform coding of noisy sources at high rates , 2004, Conference Record of the Thirty-Eighth Asilomar Conference on Signals, Systems and Computers, 2004..

[29]  L. Joseph,et al.  Bayesian Statistics: An Introduction , 1989 .

[30]  David J. Sakrison,et al.  Source encoding in the presence of random disturbance (Corresp.) , 1967, IEEE Trans. Inf. Theory.

[31]  Toby Berger,et al.  Estimation via compressed information , 1988, IEEE Trans. Inf. Theory.

[32]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[33]  Toby Berger,et al.  The quadratic Gaussian CEO problem , 1997, IEEE Trans. Inf. Theory.

[34]  Toby Berger,et al.  The CEO problem [multiterminal source coding] , 1996, IEEE Trans. Inf. Theory.

[35]  Hans S. Witsenhausen,et al.  Indirect rate distortion problems , 1980, IEEE Trans. Inf. Theory.

[36]  Kannan Ramchandran,et al.  On rate-constrained distributed estimation in unreliable sensor networks , 2005, IEEE Journal on Selected Areas in Communications.

[37]  Allen Gersho,et al.  Asymptotically optimal block quantization , 1979, IEEE Trans. Inf. Theory.

[38]  H. Vincent Poor,et al.  An Introduction to Signal Detection and Estimation , 1994, Springer Texts in Electrical Engineering.

[39]  H. Vincent Poor,et al.  An introduction to signal detection and estimation (2nd ed.) , 1994 .