On deep learning-based channel decoding

We revisit the idea of using deep neural networks for one-shot decoding of random and structured codes, such as polar codes. Although it is possible to achieve maximum a posteriori (MAP) bit error rate (BER) performance for both code families and for short codeword lengths, we observe that (i) structured codes are easier to learn and (ii) the neural network is able to generalize to codewords that it has never seen during training for structured, but not for random codes. These results provide some evidence that neural networks can learn a form of decoding algorithm, rather than only a simple classifier. We introduce the metric normalized validation error (NVE) in order to further investigate the potential and limitations of deep learning-based decoding with respect to performance and complexity.

[1]  L. G. Tallini,et al.  Neural nets for decoding error-correcting codes , 1995, IEEE Technical Applications Conference and Workshops. Northcon/95. Conference Record.

[2]  Hava T. Siegelmann,et al.  On the Computational Power of Neural Nets , 1995, J. Comput. Syst. Sci..

[3]  Erol Gelenbe,et al.  Random neural network decoder for error correcting codes , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[4]  Erdal Arikan,et al.  Channel Polarization: A Method for Constructing Capacity-Achieving Codes for Symmetric Binary-Input Memoryless Channels , 2008, IEEE Transactions on Information Theory.

[5]  Stephen B. Wicker,et al.  Path Output Register Selection Register 4 t Maximum Path Metric Selection , 2004 .

[6]  R. W. Means,et al.  Neural network error correcting decoders for block and convolutional codes , 1990, [Proceedings] GLOBECOM '90: IEEE Global Telecommunications Conference and Exhibition.

[7]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[8]  Yair Be'ery,et al.  Learning to decode linear codes using deep learning , 2016, 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[9]  G. Kane.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[10]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[11]  Gengsheng L. Zeng,et al.  An application of neural net in decoding error-correcting codes , 1989, IEEE International Symposium on Circuits and Systems,.

[12]  Yuen-Hsien Tseng,et al.  Neural Network Decoders for Linear Block Codes , 2002, Int. J. Comput. Eng. Sci..

[13]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[14]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[15]  Yoshua Bengio,et al.  Algorithms for Hyper-Parameter Optimization , 2011, NIPS.

[16]  Jehoshua Bruck,et al.  Neural networks, error-correcting codes, and polynomials over the binary n -cube , 1989, IEEE Trans. Inf. Theory.

[17]  Jukka Henriksson,et al.  A recurrent neural decoder for convolutional codes , 1999, 1999 IEEE International Conference on Communications (Cat. No. 99CH36311).

[18]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[19]  Stephen B. Wicker,et al.  An Artificial Neural Net Viterbi Decoder , 1995, Proceedings of 1995 IEEE International Symposium on Information Theory.

[20]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[21]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[22]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[23]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[24]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[25]  G. Di Cataldo,et al.  On the use of neural networks for Hamming coding , 1991, 1991., IEEE International Sympoisum on Circuits and Systems.