perm2vec: Attentive Graph Permutation Selection for Decoding of Error Correction Codes

Error correction codes are an integral part of communication applications, boosting the reliability of transmission. The optimal decoding of transmitted codewords is the maximum likelihood rule, which is NP-hard due to the curse of dimensionality. For practical realizations, sub-optimal decoding algorithms are employed; yet limited theoretical insights prevent one from exploiting the full potential of these algorithms. One such insight is the choice of permutation in permutation decoding. We present a data-driven framework for permutation selection, combining domain knowledge with machine learning concepts such as node embedding and self-attention. Significant and consistent improvements in the bit error rate are introduced for all simulated codes, over the baseline decoders. To the best of the authors’ knowledge, this work is the first to leverage the benefits of the neural Transformer networks in physical layer communication systems.

[1]  Riccardo Raheli,et al.  Reinforcement Learning for Channel Coding: Learned Bit-Flipping Decoding , 2019, 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[2]  Yoni Choukroun,et al.  Deep Learning for Decoding of Linear Codes - A Syndrome-Based Approach , 2018, 2018 IEEE International Symposium on Information Theory (ISIT).

[3]  Xiao Lin,et al.  Data-Efficient Mutual Information Neural Estimator , 2019, ArXiv.

[4]  David Burshtein,et al.  Blind Channel Equalization Using Variational Autoencoders , 2018, 2018 IEEE International Conference on Communications Workshops (ICC Workshops).

[5]  Ben Glocker,et al.  Image-and-Spatial Transformer Networks for Structure-Guided Image Registration , 2019, MICCAI.

[6]  Yair Be'ery,et al.  Improved random redundant iterative HDPC decoding , 2009, IEEE Transactions on Communications.

[7]  Warren J. Gross,et al.  Decoding Reed-Muller and Polar Codes by Successive Factor Graph Permutations , 2018, 2018 IEEE 10th International Symposium on Turbo Codes & Iterative Information Processing (ISTC).

[8]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[9]  Oren Barkan,et al.  Explainable Recommendations via Attentive Multi-Persona Collaborative Filtering , 2020, RecSys.

[10]  J. Macwilliams.  Permutation decoding of systematic codes , 1964 .

[11]  Rüdiger L. Urbanke,et al.  Polar Codes for Channel and Source Coding , 2009, ArXiv.

[12]  Yair Be'ery,et al.  Learning to decode linear codes using deep learning , 2016, 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[13]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[14]  Omer Levy,et al.  BERT for Coreference Resolution: Baselines and Analysis , 2019, EMNLP/IJCNLP.

[15]  Osvaldo Simeone,et al.  A Very Brief Introduction to Machine Learning With Applications to Communication Systems , 2018, IEEE Transactions on Cognitive Communications and Networking.

[16]  O. Antoine,et al.  Theory of Error-correcting Codes , 2022 .

[17]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[18]  David Burshtein,et al.  Deep Learning Methods for Improved Decoding of Linear Codes , 2017, IEEE Journal of Selected Topics in Signal Processing.

[19]  W. Marsden.  I and J , 2012 .

[20]  Joonhyuk Kang,et al.  From Learning to Meta-Learning: Reduced Training Overhead and Complexity for Communication Systems , 2020, 2020 2nd 6G Wireless Summit (6G SUMMIT).

[21]  Kenza Guenda,et al.  The Permutation Groups and the Equivalence of Cyclic and Quasi-Cyclic Codes , 2010, ArXiv.

[22]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[23]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[24]  Mikhail Kamenev,et al.  A New Permutation Decoding Method for Reed-Muller Codes , 2019, 2019 IEEE International Symposium on Information Theory (ISIT).

[25]  Oren Barkan,et al.  Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embedding , 2019, AAAI.

[26]  Yoshua Bengio,et al.  Mutual Information Neural Estimation , 2018, ICML.

[27]  Sreeram Kannan,et al.  Communication Algorithms via Deep Learning , 2018, ICLR.

[28]  Ahmed Elkelesh,et al.  Belief Propagation List Decoding of Polar Codes , 2018, IEEE Communications Letters.

[29]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[30]  Mérouane Debbah,et al.  Wireless Networks Design in the Era of Deep Learning: Model-Based, AI-Based, or Both? , 2019, IEEE Transactions on Communications.

[31]  Henry D. Pfister,et al.  Learned Belief-Propagation Decoding with Simple Scaling and SNR Adaptation , 2019, 2019 IEEE International Symposium on Information Theory (ISIT).

[32]  Amir H. Banihashemi,et al.  On the Tanner Graph Cycle Distribution of Random LDPC, Random Protograph-Based LDPC, and Random Quasi-Cyclic LDPC Code Ensembles , 2017, IEEE Transactions on Information Theory.

[33]  Rüdiger L. Urbanke,et al.  Modern Coding Theory , 2008 .

[34]  Neil Genzlinger.  A. and Q , 2006 .

[35]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[36]  Saif E. A. Alnawayseh,et al.  Ordered statistics-based list decoding techniques for linear binary block codes , 2012, EURASIP Journal on Wireless Communications and Networking.

[37]  Oren Barkan,et al.  Attentive Item2vec: Neural Attentive User Representations , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[38]  Yair Be'ery,et al.  Active Deep Decoding of Linear Codes , 2019, IEEE Transactions on Communications.

[39]  Yair Be'ery,et al.  RNN Decoding of Linear Block Codes , 2017, ArXiv.

[40]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[41]  Nicolas Usunier,et al.  End-to-End Object Detection with Transformers , 2020, ECCV.

[42]  Warren J. Gross,et al.  On the Decoding of Polar Codes on Permuted Factor Graphs , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[43]  Yair Be'ery,et al.  Near Maximum Likelihood Decoding with Deep Learning , 2018, ArXiv.

[44]  Yiming Yang,et al.  XLNet: Generalized Autoregressive Pretraining for Language Understanding , 2019, NeurIPS.

[45]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .