Reliability of Universal Decoding Based on Vector-Quantized Codewords

Motivated by applications of biometric identification and content identification systems, we consider the problem of random coding for channels, where each codeword undergoes vector quantization, and where the decoder bases its decision only on the compressed codewords and the channel output, which is, in turn, the channel’s response to the transmission of an original codeword, before compression. For memoryless sources and memoryless channels with finite alphabets, we propose a new universal decoder and analyze its error exponent, which improves on an earlier result by Dasarathy and Draper (2011), who used the classic maximum mutual information universal decoder. We show that our universal decoder provides the same error exponent as that of the optimal, maximum likelihood decoder, at least as long as all single-letter transition probabilities of the channel are positive.

[1]  Neri Merhav Reliability of Universal Decoding Based on Vector-Quantized Codewords , 2017, IEEE Trans. Inf. Theory.

[2]  Meir Feder,et al.  Universal Decoding for Channels with Memory , 1998, IEEE Trans. Inf. Theory.

[3]  Frans M. J. Willems,et al.  Biometric Security from an Information-Theoretical Perspective , 2012, Found. Trends Commun. Inf. Theory.

[4]  D. A. Bell,et al.  Information Theory and Reliable Communication , 1969 .

[5]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[6]  Stark C. Draper,et al.  Upper and Lower Bounds on the Reliability of Content Identification , 2014 .

[7]  Joseph A. O'Sullivan,et al.  Achievable Rates for Pattern Recognition , 2005, IEEE Transactions on Information Theory.

[8]  Stark C. Draper,et al.  On reliability of content identification from databases based on noisy queries , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[9]  Ertem Tuncel Capacity/Storage Tradeoff in High-Dimensional Identification Systems , 2006, IEEE Transactions on Information Theory.

[10]  Neri Merhav Universal Decoding for Arbitrary Channels Relative to a Given Class of Decoding Metrics , 2013, IEEE Transactions on Information Theory.

[11]  Ton Kalker,et al.  On the capacity of a biometrical identification system , 2003, IEEE International Symposium on Information Theory, 2003. Proceedings..

[12]  Neri Merhav,et al.  Statistical Physics and Information Theory , 2010, Found. Trends Commun. Inf. Theory.

[13]  Nadav Shulman,et al.  Communication over an unknown channel via common broadcasting , 2003 .

[14]  Min Wu,et al.  Modeling and analysis of content identification , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[15]  Amos Lapidoth,et al.  On the Universality of the LZ-Based Decoding Algorithm , 1998, IEEE Trans. Inf. Theory.

[16]  Robert G. Gallager,et al.  The random coding bound is tight for the average code (Corresp.) , 1973, IEEE Trans. Inf. Theory.

[17]  Neri Merhav Universal Decoding Using a Noisy Codebook , 2018, IEEE Transactions on Information Theory.