Universal decoding using a noisy codebook

We consider the topic of universal decoding with a decoder that does not have direct access to the codebook, but only to noisy versions of the various randomly generated codewords, a problem motivated by biometrical identification systems. Both the source that generates the original (clean) codewords, and the channel that corrupts them in generating the noisy codewords, as well as the main channel for communicating the messages, are all modeled by non-unifilar, finite-state systems (hidden Markov models). As in previous works on universal decoding, here too, the average error probability of our proposed universal decoder is shown to be as small as that of the optimal maximum likelihood (ML) decoder, up to a multiplicative factor that is a sub-exponential function of the block length. It therefore has the same error exponent, whenever the ML decoder has a positive error exponent. The universal decoding metric is based on Lempel-Ziv (LZ) incremental parsing of each noisy codeword jointly with the given channel output vector, but this metric is somewhat different from the one proposed in earlier works on universal decoding for finite-state channels, by Ziv (1985) and by Lapidoth and Ziv (1998). The reason for the difference is that here, unlike in those earlier works, the probability distribution that governs the (noisy) codewords is, in general, not uniform across its support. This non-uniformity of the codeword distribution also makes our derivation more challenging. Another reason for the more challenging analysis is the fact that the effective induced channel between the noisy codeword of the transmitted message and the main channel output is not a finite-state channel in general.

[1]  Frans M. J. Willems,et al.  Biometric Security from an Information-Theoretical Perspective , 2012, Found. Trends Commun. Inf. Theory.

[2]  Jacob Ziv,et al.  Universal decoding for finite-state channels , 1985, IEEE Trans. Inf. Theory.

[3]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[4]  Stark C. Draper,et al.  On reliability of content identification from databases based on noisy queries , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[5]  T Petrie,et al.  Probabilistic functions of finite-state markov chains. , 1967, Proceedings of the National Academy of Sciences of the United States of America.

[6]  Neri Merhav Universal decoding for memoryless Gaussian channels with a deterministic interference , 1993, IEEE Trans. Inf. Theory.

[7]  Neri Merhav Universal Decoding for Arbitrary Channels Relative to a Given Class of Decoding Metrics , 2013, IEEE Transactions on Information Theory.

[8]  Neri Merhav,et al.  Hidden Markov processes , 2002, IEEE Trans. Inf. Theory.

[9]  Ertem Tuncel Capacity/Storage Tradeoff in High-Dimensional Identification Systems , 2006, ISIT.

[10]  Neri Merhav,et al.  Estimating the number of states of a finite-state source , 1992, IEEE Trans. Inf. Theory.

[11]  Ton Kalker,et al.  On the capacity of a biometrical identification system , 2003, IEEE International Symposium on Information Theory, 2003. Proceedings..

[12]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[13]  Amos Lapidoth,et al.  On the Universality of the LZ-Based Decoding Algorithm , 1998, IEEE Trans. Inf. Theory.

[14]  Neri Merhav,et al.  Universal composite hypothesis testing: A competitive minimax approach , 2002, IEEE Trans. Inf. Theory.

[15]  Meir Feder,et al.  Communication over Individual Channels -- a general framework , 2012, ArXiv.

[16]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[17]  Meir Feder,et al.  Universal Decoding for Channels with Memory , 1998, IEEE Trans. Inf. Theory.

[18]  Imre Csiszár Linear codes for sources and source networks: Error exponents, universal coding , 1982, IEEE Trans. Inf. Theory.

[19]  Meir Feder,et al.  Communication Over Individual Channels , 2009, IEEE Transactions on Information Theory.

[20]  L. Baum,et al.  Statistical Inference for Probabilistic Functions of Finite State Markov Chains , 1966 .

[21]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .