Learning to Denoise and Decode: A Novel Residual Neural Network Decoder for Polar Codes

Polar codes are known as the first capacity-achievable codes with low encoding and decoding complexity. The sequential decoding nature of traditional polar decoding algorithms such as successive cancellation (SC) results in high decoding latency, which is not suitable for services that require high reliability and low latency. Deep learning for decoding, referred to as neural network decoder (NND), has shown strong competitiveness because of its non-iterative and full-parallel feature. Whereas, the bit-error-rate (BER) performance of NND is still not satisfactory. In this paper, we first propose a residual learning denoiser (RLD) for polar codes. The RLD can remarkably improve the signal-to-noise ratio (SNR) and reduce the symbol-error-rate (SER) of received symbols. In order to decode polar codes more efficiently, we subsequently propose a residual neural network decoder (RNND) for polar codes. Different from the traditional pure-NND (PNND) which directly uses neural network for decoding received symbols, the proposed RNND concatenates a RLD used for denoising and a NND used for decoding. We provide a novel multi-task learning (MTL) strategy to jointly optimize the denoiser and decoder, and find that the denoising gain of Joint-RLD is more significant than the Independent-RLD. Numerical results show that the proposed RNND outperforms its counterpart PNND with regard to the BER performance. In addition, the optimal RNND(MLP-MLP) approaches the traditional SC decoding performance, while saving more than one hundred times of computation time. Eventually, scalability of the RNND to longer polar codes as well as LDPC codes further demonstrates the superiority of our proposed scheme.

[1]  Stephan ten Brink,et al.  Scaling Deep Learning-Based Decoding of Polar Codes via Partitioning , 2017, GLOBECOM 2017 - 2017 IEEE Global Communications Conference.

[2]  Juyul Lee,et al.  Decoding of Polar Code by Using Deep Feed-Forward Neural Networks , 2018, 2018 International Conference on Computing, Networking and Communications (ICNC).

[3]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[4]  Xiaohu You,et al.  Improved polar decoder based on deep learning , 2017, 2017 IEEE International Workshop on Signal Processing Systems (SiPS).

[5]  Jean Ponce,et al.  A Theoretical Analysis of Feature Pooling in Visual Recognition , 2010, ICML.

[6]  Xiaohu You,et al.  Polar Decoding on Sparse Graphs with Deep Learning , 2018, 2018 52nd Asilomar Conference on Signals, Systems, and Computers.

[7]  Erdal Arikan,et al.  Channel Polarization: A Method for Constructing Capacity-Achieving Codes for Symmetric Binary-Input Memoryless Channels , 2008, IEEE Transactions on Information Theory.

[8]  Philip Schniter,et al.  Onsager-corrected deep learning for sparse linear inverse problems , 2016, 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[9]  Stephen B. Wicker,et al.  Path Output Register Selection Register 4 t Maximum Path Metric Selection , 2004 .

[10]  Jaehoon Lee,et al.  Wide neural networks of any depth evolve as linear models under gradient descent , 2019, NeurIPS.

[11]  Jakob Hoydis,et al.  An Introduction to Deep Learning for the Physical Layer , 2017, IEEE Transactions on Cognitive Communications and Networking.

[12]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[13]  G. Di Cataldo,et al.  On the use of neural networks for Hamming coding , 1991, 1991., IEEE International Sympoisum on Circuits and Systems.

[14]  Geoffrey Ye Li,et al.  Initial Results on Deep Learning for Joint Channel Equalization and Decoding , 2017, 2017 IEEE 86th Vehicular Technology Conference (VTC-Fall).

[15]  Yair Be'ery,et al.  Learning to decode linear codes using deep learning , 2016, 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[16]  K. Niu,et al.  Stack decoding of polar codes , 2012 .

[17]  Jianxiong Xiao,et al.  DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  Rich Caruana,et al.  Multitask Learning , 1997, Machine Learning.

[19]  Yoshua Bengio,et al.  Gated Feedback Recurrent Neural Networks , 2015, ICML.

[20]  A. R. Karami,et al.  Multi Layer Perceptron Neural Networks Decoder for LDPC Codes , 2009, 2009 5th International Conference on Wireless Communications, Networking and Mobile Computing.

[21]  Louis B. Rall,et al.  Automatic differentiation , 1981 .

[22]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[23]  Peter Trifonov,et al.  Efficient Design and Decoding of Polar Codes , 2012, IEEE Transactions on Communications.

[24]  Shugong Xu,et al.  A Unified Deep Learning Based Polar-LDPC Decoder for 5G Communication Systems , 2018, 2018 10th International Conference on Wireless Communications and Signal Processing (WCSP).

[25]  Kai Chen,et al.  CRC-Aided Decoding of Polar Codes , 2012, IEEE Communications Letters.

[26]  Radford M. Neal,et al.  Near Shannon limit performance of low density parity check codes , 1996 .

[27]  Juyul Lee,et al.  Activation functions of deep neural networks for polar decoding applications , 2017, 2017 IEEE 28th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC).

[28]  Kai Lai Chung,et al.  A Course in Probability Theory , 1949 .

[29]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[30]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[31]  Alexander Vardy,et al.  List decoding of polar codes , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[32]  Guan Gui,et al.  Deep Learning for Super-Resolution Channel Estimation and DOA Estimation Based Massive MIMO System , 2018, IEEE Transactions on Vehicular Technology.

[33]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[34]  S. Shelke,et al.  A novel multistage classification and Wavelet based kernel generation for handwritten Marathi compound character recognition , 2011, 2011 International Conference on Communications and Signal Processing.

[35]  K. Niu,et al.  List successive cancellation decoding of polar codes , 2012 .

[36]  Ade Irawan,et al.  Deep Learning for Polar Codes over Flat Fading Channels , 2019, 2019 International Conference on Artificial Intelligence in Information and Communication (ICAIIC).

[37]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[38]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[39]  Huazi Zhang,et al.  Performance Evaluation of Channel Decoding with Deep Neural Networks , 2017, 2018 IEEE International Conference on Communications (ICC).

[40]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[41]  An-Yeu Wu,et al.  Low-complexity Recurrent Neural Network-based Polar Decoder with Weight Quantization Mechanism , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[42]  Mustafa ElNainay,et al.  Deep Convolutional Neural Network with Multi-Task Learning Scheme for Modulations Recognition , 2019, 2019 15th International Wireless Communications & Mobile Computing Conference (IWCMC).

[43]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Yair Be'ery,et al.  RNN Decoding of Linear Block Codes , 2017, ArXiv.