Bounds on information combining for parity-check equations

When several independent channels are coupled by a parity check constraint on their inputs, the mutual information between the input of one channel and the outputs of all other channels can be expressed as a combination of the mutual information between the input and the output of each individual channel. This concept is denoted as information combining. For binary-input symmetric discrete memoryless channels, we present bounds on the combined information which are only based on the mutual information of the channels. Furthermore, we show that these bounds cannot be further improved. Exact expressions are provided for the case that all channels are binary symmetric channels and for the case that all channels are binary erasure channels.

[1]  Hans-Andrea Loeliger,et al.  Codes and iterative decoding on general graphs , 1995, Eur. Trans. Telecommun..

[2]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[3]  Alain Glavieux,et al.  Reflections on the Prize Paper : "Near optimum error-correcting coding and decoding: turbo codes" , 1998 .

[4]  Dariush Divsalar,et al.  Serial Concatenation of Interleaved Codes: Performance Analysis, Design, and Iterative Decoding , 1997, IEEE Trans. Inf. Theory.

[5]  Robert F. H. Fischer,et al.  Information processing in soft-output decoding , 2001 .

[6]  Aaron D. Wyner,et al.  A theorem on the entropy of certain binary sequences and applications-I , 1973, IEEE Trans. Inf. Theory.

[7]  David J. C. MacKay,et al.  Good Error-Correcting Codes Based on Very Sparse Matrices , 1997, IEEE Trans. Inf. Theory.

[8]  Robert G. Gallager Low Density Parity Check Code , 1962 .

[9]  Johannes B. Huber,et al.  Bounds on information combining , 2005, IEEE Transactions on Information Theory.

[10]  John Cocke,et al.  Optimal decoding of linear codes for minimizing symbol error rate (Corresp.) , 1974, IEEE Trans. Inf. Theory.

[11]  Stephan ten Brink,et al.  Convergence behavior of iteratively decoded parallel concatenated codes , 2001, IEEE Trans. Commun..

[12]  Rolf Johannesson,et al.  Soft-output-decoding: Some aspects from information theory , 2002 .

[13]  Aaron D. Wyner,et al.  A theorem on the entropy of certain binary sequences and applications-II , 1973, IEEE Trans. Inf. Theory.

[14]  Sergio Benedetto,et al.  Unveiling turbo codes: some results on parallel concatenated coding schemes , 1996, IEEE Trans. Inf. Theory.

[15]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[16]  Joachim Hagenauer,et al.  Iterative decoding of binary block and convolutional codes , 1996, IEEE Trans. Inf. Theory.

[17]  Robert Michael Tanner,et al.  A recursive approach to low complexity codes , 1981, IEEE Trans. Inf. Theory.