List Decoding - Random Coding Exponents and Expurgated Exponents

Some new results are derived concerning random coding error exponents and expurgated exponents for list decoding with a deterministic list-size \(L\) . Two asymptotic regimes are considered, the fixed list-size regime, where \(L\) is fixed independently of the block length \(n\) , and exponential list size, where \(L\) grows exponentially with \(n\) . We first derive a general upper bound on the list-decoding average error probability, which is suitable for both regimes. This bound leads to more specific bounds in the two regimes. In the fixed list-size regime, the bound is related to known bounds and we establish its exponential tightness. In the exponential list-size regime, we establish the achievability of the well-known sphere packing lower bound. An immediate byproduct of our analysis in both regimes is the universality of the maximum mutual information list decoder in the error exponent sense. Finally, we consider expurgated bounds at low rates, both using Gallager’s approach and Csiszar–Korner–Marton approach, which is equivalent for the optimal input assignment. The latter expurgated bound, which involves the notion of multi-information, is also modified to apply to continuous alphabet channels, and in particular, to the Gaussian memoryless channel, where the expression of the expurgated bound becomes quite explicit.

[1]  Peter Elias,et al.  List decoding for noisy channels , 1957 .

[2]  N. Seshadri,et al.  Generalized Viterbi algorithms for error detection with convolutional codes , 1989, IEEE Global Telecommunications Conference, 1989, and Exhibition. 'Communications Technology for the 1990s and Beyond.

[3]  Neri Merhav,et al.  Guessing Subject to Distortion , 1998, IEEE Trans. Inf. Theory.

[4]  Carl-Erik W. Sundberg,et al.  List Viterbi decoding algorithms with applications , 1994, IEEE Trans. Commun..

[5]  Andrew J. Viterbi,et al.  Principles of Digital Communication and Coding , 1979 .

[6]  Neri Merhav,et al.  Joint Source-Channel Coding and Guessing with Application to Sequential Decoding , 1998, IEEE Trans. Inf. Theory.

[7]  Neri Merhav,et al.  Asymptotically Optimal Decision Rules for Joint Detection and Source Coding , 2013, IEEE Transactions on Information Theory.

[8]  Vladimir M. Blinovsky Error Probability Exponent of List Decoding at Low Rates , 2001, Probl. Inf. Transm..

[9]  Vladimir M. Blinovsky,et al.  List decoding , 1992, Discrete Mathematics.

[10]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[11]  Neri Merhav,et al.  Statistical Physics and Information Theory , 2010, Found. Trends Commun. Inf. Theory.

[12]  Robert G. Gallager,et al.  The random coding bound is tight for the average code (Corresp.) , 1973, IEEE Trans. Inf. Theory.

[13]  Neri Merhav,et al.  List Decoding—Random Coding Exponents and Expurgated Exponents , 2013, IEEE Transactions on Information Theory.

[14]  Neri Merhav,et al.  Codeword or noise? Exact random coding exponents for slotted asynchronism , 2014, 2014 IEEE International Symposium on Information Theory.

[15]  Neri Merhav Universal decoding for memoryless Gaussian channels with a deterministic interference , 1993, IEEE Trans. Inf. Theory.

[16]  Elwyn R. Berlekamp,et al.  Lower Bounds to Error Probability for Coding on Discrete Memoryless Channels. II , 1967, Inf. Control..

[17]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[18]  Shlomo Shamai,et al.  Performance Bounds for Erasure, List, and Decision Feedback Schemes With Linear Block Codes , 2010, IEEE Transactions on Information Theory.

[19]  Emre Telatar,et al.  Exponential bounds for list size moments and error probability , 1998, 1998 Information Theory Workshop (Cat. No.98EX131).

[20]  Ivan J. Fair,et al.  Improved Analysis of List Decoding and Its Application to Convolutional Codes and Turbo Codes , 2007, IEEE Transactions on Information Theory.

[21]  G. David Forney,et al.  Exponential error bounds for erasure, list, and decision feedback schemes , 1968, IEEE Trans. Inf. Theory.

[22]  Venkatesan Guruswami,et al.  List decoding of error correcting codes , 2001 .

[23]  R. Gallager Information Theory and Reliable Communication , 1968 .

[24]  M. Studený,et al.  The Multiinformation Function as a Tool for Measuring Stochastic Dependence , 1998, Learning in Graphical Models.

[25]  Volodia Blinovsky Multiple Packing of the Euclidean Sphere , 1999, IEEE Trans. Inf. Theory.

[26]  Neri Merhav On random coding error exponents of watermarking systems , 2000, IEEE Trans. Inf. Theory.

[27]  Emre Telatar Zero-error list capacities of discrete memoryless channels , 1997, IEEE Trans. Inf. Theory.