Deep Learning-Based Belief Propagation Algorithm over Non-Binary Finite Fields

In this paper, we present a deep learning method for improving the belief propagation algorithm over non-binary Galois fields. The method is based on the deep unfolding structure of the factor graph representing LDPC matrix where the inference iterations according to Hadamard transform belief propagation algorithm are translated into layers of the neural networks. Afterwards, we introduce genetic algorithm to optimize the weights of the neural networks. The trained neural networks render much better performances than conventional belief propagation algorithms for both decoding of linear block codes and the compressed sensing reconstructions. The loss in terms of training process are also depicted, which show that the algorithm can converge after acceptable times of training.

[1]  J. McCall,et al.  Genetic algorithms for modelling and optimisation , 2005 .

[2]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[3]  Nikos D. Sidiropoulos,et al.  Learning to optimize: Training deep neural networks for wireless resource management , 2017, 2017 IEEE 18th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[4]  David J. C. MacKay,et al.  Low-density parity check codes over GF(q) , 1998, IEEE Communications Letters.

[5]  Randall S. Sexton,et al.  Toward global optimization of neural networks: A comparison of the genetic algorithm and backpropagation , 1998, Decis. Support Syst..

[6]  Sreeram Kannan,et al.  Communication Algorithms via Deep Learning , 2018, ICLR.

[7]  Tadashi Wadayama,et al.  Deep Learning-Aided Trainable Projected Gradient Decoding for LDPC Codes , 2019, 2019 IEEE International Symposium on Information Theory (ISIT).

[8]  Fan Li,et al.  Efficient and Robust Distributed Digital Codec Framework for Jointly Sparse Correlated Signals , 2019, IEEE Access.

[9]  Bane V. Vasic,et al.  Learning to Decode LDPC Codes with Finite-Alphabet Message Passing , 2018, 2018 Information Theory and Applications Workshop (ITA).

[10]  Jonathan Le Roux,et al.  Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures , 2014, ArXiv.

[11]  Andrea J. Goldsmith,et al.  Deep Learning for Joint Source-Channel Coding of Text , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Deniz Gündüz,et al.  Deep Joint Source-Channel Coding for Wireless Image Transmission , 2019, IEEE Transactions on Cognitive Communications and Networking.

[13]  Lawrence Davis,et al.  Training Feedforward Neural Networks Using Genetic Algorithms , 1989, IJCAI.

[14]  Stephan ten Brink,et al.  On deep learning-based channel decoding , 2017, 2017 51st Annual Conference on Information Sciences and Systems (CISS).

[15]  Robert E. Dorsey,et al.  Genetic algorithms for estimation problems with multiple optima , 1995 .

[16]  Jin-Taek Seong,et al.  Necessary and Sufficient Conditions for Recovery of Sparse Signals over Finite Fields , 2013, IEEE Communications Letters.

[17]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[18]  D. Declercq,et al.  Fast Decoding Algorithm for LDPC over GF(2q) , 2003 .

[19]  Yair Be'ery,et al.  Learning to decode linear codes using deep learning , 2016, 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[20]  Yair Be'ery,et al.  Active Deep Decoding of Linear Codes , 2019, IEEE Transactions on Communications.