Feedback and belief propagation

We demonstrate that feedback in discrete memoryless channels has the capability of greatly lowering the block error rate of codes designed for open-loop operation. First we show how to use full feedback of the channel output to turn any capacity achieving code into a reliability-function achieving code. Second, we propose a practical embodiment based on sparse-graph codes, belief propagation, and a variation of the closed-loop iterative doping algorithm. This scheme takes advantage of any available limited-rate feedback to bootstrap good block error rate from good bit error rate.

[1]  Anthony J. Kramer,et al.  Improving communication reliability by use of an intermittent feedback channel , 1969, IEEE Trans. Inf. Theory.

[2]  J.M. Ooi,et al.  Fast Iterative Coding Techniques for Feedback Channels , 1998, IEEE Trans. Inf. Theory.

[3]  Michael Horstein,et al.  Sequential transmission using noiseless feedback , 1963, IEEE Trans. Inf. Theory.

[4]  Giuseppe Caire,et al.  Almost noiseless joint source channel coding-decoding of sources of memory , 2004 .

[5]  J. Pieter M. Schalkwijk.  A class of simple and optimal strategies for block coding on the binary symmetric channel with noiseless feedback , 1971, IEEE Trans. Inf. Theory.

[6]  Richard E. Blahut,et al.  Principles and practice of information theory , 1987 .

[7]  Aaron D. Wyner,et al.  On the Schalkwijk-Kailath coding scheme with a peak energy constraint , 1968, IEEE Trans. Inf. Theory.

[8]  Thomas Kailath,et al.  A coding scheme for additive noise channels with feedback-I: No bandwidth constraint , 1966, IEEE Trans. Inf. Theory.

[9]  Claude E. Shannon,et al.  The zero error capacity of a noisy channel , 1956, IRE Trans. Inf. Theory.

[10]  Kohji Itoh,et al.  Asymptotic performance of a modified Schalkwijk-Barron scheme for channels with noiseless feedback (Corresp.) , 1979, IEEE Trans. Inf. Theory.

[11]  M. V. Burnašev.  SEQUENTIAL DISCRIMINATION OF HYPOTHESES WITH CONTROL OF OBSERVATIONS , 1980 .

[12]  Shlomo Shamai,et al.  A new data compression algorithm for sources with memory based on error correcting codes , 2003, Proceedings 2003 IEEE Information Theory Workshop (Cat. No.03EX674).

[13]  Rudolf Ahlswede,et al.  A constructive proof of the coding theorem for discrete memoryless channels with feedback , 1973 .

[14]  Alexander Barg,et al.  Error exponents of expander codes , 2002, IEEE Trans. Inf. Theory.

[15]  Shlomo Shamai,et al.  An efficient scheme for reliable error correction with limited feedback , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[16]  Emre Telatar,et al.  On the universality of Burnashev's error exponent , 2005, IEEE Transactions on Information Theory.

[17]  Rüdiger L. Urbanke,et al.  Capacity-achieving ensembles for the binary erasure channel with bounded complexity , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[18]  J. Pieter M. Schalkwijk,et al.  Sequential signalling under a peak power constraint , 1971, IEEE Trans. Inf. Theory.

[19]  J. Pieter M. Schalkwijk,et al.  On the error probability for a class of binary recursive feedback strategies , 1973, IEEE Trans. Inf. Theory.

[20]  Giuseppe Caire.  Universal data compression with LDPC codes , 2003 .

[21]  G. David Forney,et al.  Exponential error bounds for erasure, list, and decision feedback schemes , 1968, IEEE Trans. Inf. Theory.

[22]  S. Verdú,et al.  Noiseless Data Compression with Low-Density Parity-Check Codes , 2003, Advances in Network Information Theory.

[23]  S. Shamai,et al.  Lossless data compression with error correcting codes , 2003, IEEE International Symposium on Information Theory, 2003. Proceedings..

[24]  Kannan Ramchandran,et al.  Attaining Maximal Reliability with Minimal Feedback via Joint Channel-code and Hash-Function Design , 2005 .