Universal Decoding for Arbitrary Channels Relative to a Given Class of Decoding Metrics

We consider the problem of universal decoding for arbitrary, finite-alphabet unknown channels in the random coding regime. For a given random coding distribution and a given class of metric decoders, we propose a generic universal decoder whose average error probability is, within a subexponential multiplicative factor, no larger than that of the best decoder within this class of decoders. Since the optimum, maximum likelihood (ML) decoder of the underlying channel is not necessarily assumed to belong to the given class of decoders, this setting suggests a common generalized framework for: 1) mismatched decoding, 2) universal decoding for a given family of channels, and 3) universal coding and decoding for deterministic channels using the individual sequence approach. The proof of our universality result is fairly simple, and it is demonstrated how some earlier results on universal decoding are obtained as special cases. We also demonstrate how our method extends to more complicated scenarios, like incorporation of noiseless feedback, the multiple access channel, and continuous alphabet channels.

[1]  Albert Guillén i Fàbregas,et al.  An achievable error exponent for the mismatched multiple-access channel , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[2]  Dominique de Caen,et al.  A lower bound on the probability of a union , 1997, Discret. Math..

[3]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[4]  Brian L. Hughes,et al.  A new universal random coding bound for the multiple-access channel , 1996, IEEE Trans. Inf. Theory.

[5]  Neri Merhav,et al.  Universal composite hypothesis testing: A competitive minimax approach , 2002, IEEE Trans. Inf. Theory.

[6]  Meir Feder,et al.  Communication over Individual Channels -- a general framework , 2012, ArXiv.

[7]  Neri Merhav,et al.  Achievable Error Exponents for the Private Fingerprinting Game , 2007, IEEE Transactions on Information Theory.

[8]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[9]  Meir Feder,et al.  Universal Decoding for Channels with Memory , 1998, IEEE Trans. Inf. Theory.

[10]  Neri Merhav Universal Decoding for Arbitrary Channels Relative to a Given Class of Decoding Metrics , 2014 .

[11]  Thomas M. Cover,et al.  Elements of Information Theory: Cover/Elements of Information Theory, Second Edition , 2005 .

[12]  Imre Csiszár,et al.  Channel capacity for a given decoding metric , 1995, IEEE Trans. Inf. Theory.

[13]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[14]  Jacob Ziv,et al.  Universal decoding for finite-state channels , 1985, IEEE Trans. Inf. Theory.

[15]  Meir Feder,et al.  Communicating using feedback over a binary channel with arbitrary noise sequence , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[16]  Joseph Y. N. Hui,et al.  Fundamental issues of multiple accessing , 1983 .

[17]  Imre Csiszár Linear codes for sources and source networks: Error exponents, universal coding , 1982, IEEE Trans. Inf. Theory.

[18]  Shlomo Shamai,et al.  On information rates for mismatched decoders , 1994, IEEE Trans. Inf. Theory.

[19]  Meir Feder,et al.  Universal Communication over Modulo-additive Channels with an Individual Noise Sequence , 2010, ArXiv.

[20]  Meir Feder,et al.  Universal Communication—Part I: Modulo Additive Channels , 2013, IEEE Transactions on Information Theory.

[21]  Imre Csiszár Generalized cutoff rates and Renyi's information measures , 1995, IEEE Trans. Inf. Theory.

[22]  Tsachy Weissman,et al.  The porosity of additive noise sequences , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[23]  Amos Lapidoth,et al.  On the Universality of the LZ-Based Decoding Algorithm , 1998, IEEE Trans. Inf. Theory.

[24]  Neri Merhav Universal detection of messages via finite-state channels , 2000, IEEE Trans. Inf. Theory.

[25]  Hans-Martin Wallmeier,et al.  Random coding bound and codes produced by permutations for the multiple-access channel , 1985, IEEE Trans. Inf. Theory.

[26]  Nadav Shulman,et al.  Communication over an unknown channel via common broadcasting , 2003 .

[27]  Neri Merhav Universal decoding for memoryless Gaussian channels with a deterministic interference , 1993, IEEE Trans. Inf. Theory.