Adaptive Learning Vector Quantization for Online Parametric Estimation

This paper addresses the problem of parameter estimation in a quantized and online setting. A sensing unit collects random vector-valued samples from the environment. These samples are quantized and transmitted to a central processor which generates an online estimate of the unknown parameter. This paper provides a closed-form expression of the excess mean square error (MSE) caused by quantization in the high-rate regime i.e., when the number of quantization levels is supposed to be large. Next, we determine the quantizers which mitigate the excess MSE. The optimal quantization rule unfortunately depends on the unknown parameter. To circumvent this issue, we introduce a novel adaptive learning vector quantization scheme which allows to simultaneously estimate the parameter of interest and select an efficient quantizer.

[1]  Saleem A. Kassam,et al.  Optimum Quantization for Signal Detection , 1977, IEEE Trans. Commun..

[2]  Pascal Bianchi,et al.  High-Rate Vector Quantization for the Neyman–Pearson Detection of Correlated Processes , 2010, IEEE Transactions on Information Theory.

[3]  D. Titterington.  Recursive Parameter Estimation Using Incomplete Data , 1984 .

[4]  Vivek K. Goyal,et al.  Distributed Scalar Quantization for Computing: High-Resolution Analysis and Extensions , 2008, IEEE Transactions on Information Theory.

[5]  John N. Tsitsiklis,et al.  Decentralized detection by a large number of sensors , 1988, Math. Control. Signals Syst..

[6]  J. Norris.  Appendix: probability and measure , 1997 .

[7]  Harold J. Kushner,et al.  wchastic. approximation methods for constrained and unconstrained systems , 1978 .

[8]  Patrick Duvaut,et al.  Optimum quantization for detection , 1988, IEEE Trans. Commun..

[9]  Shun-ichi Amari,et al.  Statistical Inference Under Multiterminal Data Compression , 1998, IEEE Trans. Inf. Theory.

[10]  David L. Neuhoff,et al.  Bennett's integral for vector quantizers , 1995, IEEE Trans. Inf. Theory.

[11]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[12]  Josef Hofbauer,et al.  Stochastic Approximations and Differential Inclusions , 2005, SIAM J. Control. Optim..

[13]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[14]  Meir Feder,et al.  On lattice quantization noise , 1996, IEEE Trans. Inf. Theory.

[15]  K. Lange.  A gradient algorithm locally equivalent to the EM algorithm , 1995 .

[16]  H. Poor,et al.  Fine quantization in signal detection and estimation , 1988, IEEE Trans. Inf. Theory.

[17]  Alfred O. Hero,et al.  High-rate vector quantization for detection , 2003, IEEE Trans. Inf. Theory.

[18]  Allen Gersho,et al.  Asymptotically optimal block quantization , 1979, IEEE Trans. Inf. Theory.

[19]  G.B. Giannakis,et al.  Distributed compression-estimation using wireless sensor networks , 2006, IEEE Signal Processing Magazine.

[20]  H. V. Poor,et al.  Applications of Ali-Silvey Distance Measures in the Design of Generalized Quantizers for Binary Decision Systems , 1977, IEEE Trans. Commun..

[21]  B. Delyon.  Stochastic Approximation with Decreasing Gain : Convergence and Asymptotic Theory , 2000 .

[22]  John N. Tsitsiklis,et al.  Extremal properties of likelihood-ratio quantizers , 1993, IEEE Trans. Commun..

[23]  Boris Polyak,et al.  Acceleration of stochastic approximation by averaging , 1992 .

[24]  M. Pelletier.  Weak convergence rates for stochastic approximation with application to multiple targets and simulated annealing , 1998 .

[25]  Allen Gersho,et al.  Competitive learning and soft competition for vector quantizer design , 1992, IEEE Trans. Signal Process..

[26]  Nils Sandell,et al.  Detection with Distributed Sensors , 1980, IEEE Transactions on Aerospace and Electronic Systems.

[27]  David L. Neuhoff,et al.  Quantization , 2022, IEEE Trans. Inf. Theory.

[28]  W. R. Bennett,et al.  Spectra of quantized signals , 1948, Bell Syst. Tech. J..

[29]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[30]  O. Cappé,et al.  On‐line expectation–maximization algorithm for latent data models , 2009 .

[31]  Robert M. Gray,et al.  Bayes risk weighted vector quantization with posterior estimation for image compression and classification , 1996, IEEE Trans. Image Process..