Multi Variable-layer Neural Networks for Decoding Linear Codes

The belief propagation algorithm is a state of the art decoding technique for a variety of linear codes such as LDPC codes. The iterative structure of this algorithm is reminiscent of a neural network with multiple layers. Indeed, this similarity has been recently exploited to improve the decoding performance by tuning the weights of the equivalent neural network. In this paper, we introduce a new network architecture by increasing the number of variable-node layers, while keeping the check-node layers unchanged. The changes are applied in a manner that the decoding performance of the network becomes independent of the transmitted codeword; hence, a training stage with only the all-zero codeword shall be sufficient. Simulation results on a number of well-studied linear codes, besides an improvement in the decoding performance, indicate that the new architecture is also simpler than some of the existing decoding networks.

[1]  Yuen-Hsien Tseng,et al.  Neural Network Decoders for Linear Block Codes , 2002, Int. J. Comput. Eng. Sci..

[2]  Warren J. Gross,et al.  Neural offset min-sum decoding , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[3]  Yair Be'ery,et al.  Learning to decode linear codes using deep learning , 2016, 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[4]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[5]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[6]  Amin Shokrollahi,et al.  LDPC Codes: An Introduction , 2004 .

[7]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[8]  Bane V. Vasic,et al.  Learning to Decode LDPC Codes with Finite-Alphabet Message Passing , 2018, 2018 Information Theory and Applications Workshop (ITA).

[9]  Jehoshua Bruck,et al.  Neural networks, error-correcting codes, and polynomials over the binary n -cube , 1989, IEEE Trans. Inf. Theory.

[10]  Judea Pearl,et al.  Reverend Bayes on Inference Engines: A Distributed Hierarchical Approach , 1982, AAAI.

[11]  Lior Wolf,et al.  Hyper-Graph-Network Decoders for Block Codes , 2019, NeurIPS.

[12]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[13]  Lior Wolf,et al.  Electric Analog Circuit Design with Hypernetworks And A Differential Simulator , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Ja-Ling Wu,et al.  High-order perceptrons for decoding error-correcting codes , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[15]  Robert Michael Tanner,et al.  A recursive approach to low complexity codes , 1981, IEEE Trans. Inf. Theory.

[16]  David Burshtein,et al.  Deep Learning Methods for Improved Decoding of Linear Codes , 2017, IEEE Journal of Selected Topics in Signal Processing.