Design of Communication Systems Using Deep Learning: A Variational Inference Perspective

Recent research in the design of end to end communication system using deep learning has produced models which can outperform traditional communication schemes. Most of these architectures leveraged autoencoders to design the encoder at the transmitter and decoder at the receiver and train them jointly by modeling transmit symbols as latent codes from the encoder. However, in communication systems, the receiver has to work with noise corrupted versions of transmit symbols. Traditional autoencoders are not designed to work with latent codes corrupted with noise. In this work, we provide a framework to design end to end communication systems which accounts for the existence of noise corrupted transmit symbols. The proposed method uses deep neural architecture. An objective function for optimizing these models is derived based on the concepts of variational inference. Further, domain knowledge such as channel type can be systematically integrated into the objective. Through numerical simulation, the proposed method is shown to consistently produce models with better packing density and achieving it faster in multiple popular channel models as compared to the previous works leveraging deep learning models.

[1]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[2]  Frank Nielsen,et al.  A closed-form formula for the Kullback-Leibler divergence between Cauchy distributions , 2019, ArXiv.

[3]  Mohamed-Slim Alouini,et al.  Symbol Error Rate of MPSK Over EGK Channels Perturbed by a Dominant Additive Laplacian Noise , 2015, IEEE Transactions on Communications.

[4]  Dana H. Ballard,et al.  Modular Learning in Neural Networks , 1987, AAAI.

[5]  Woongsup Lee,et al.  A Novel PAPR Reduction Scheme for OFDM System Based on Deep Learning , 2018, IEEE Communications Letters.

[6]  Polina Bayvel,et al.  End-to-End Deep Learning of Optical Fiber Communications , 2018, Journal of Lightwave Technology.

[7]  Stephan ten Brink,et al.  Deep Learning Based Communication Over the Air , 2017, IEEE Journal of Selected Topics in Signal Processing.

[8]  David M. Blei,et al.  Variational Inference: A Review for Statisticians , 2016, ArXiv.

[9]  Biing-Hwang Juang,et al.  Channel Agnostic End-to-End Learning Based Communication Systems with Conditional GAN , 2018, 2018 IEEE Globecom Workshops (GC Wkshps).

[10]  G.R. Arce,et al.  Zero-Order Statistics: A Mathematical Framework for the Processing and Characterization of Very Impulsive Signals , 2006, IEEE Transactions on Signal Processing.

[11]  Timothy J. O'Shea,et al.  Physical Layer Communications System Design Over-the-Air Using Adversarial Networks , 2018, 2018 26th European Signal Processing Conference (EUSIPCO).

[12]  Chrysostomos L. Nikias,et al.  Incoherent receivers in alpha-stable impulsive noise , 1995, IEEE Trans. Signal Process..

[13]  Nam-I Kim,et al.  Deep Learning-Aided SCMA , 2018, IEEE Communications Letters.

[14]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[15]  Eric Granger,et al.  Constrained‐CNN losses for weakly supervised segmentation☆ , 2018, Medical Image Anal..

[16]  Geoffrey E. Hinton,et al.  Autoencoders, Minimum Description Length and Helmholtz Free Energy , 1993, NIPS.

[17]  Marco Di Renzo,et al.  Molecular Communications: Model-Based and Data-Driven Receiver Design and Optimization , 2019, IEEE Access.

[18]  Ami Wiesel,et al.  Learning to Detect , 2018, IEEE Transactions on Signal Processing.

[19]  Mohamed-Slim Alouini,et al.  On the symbol error rate of M-ary MPSK over generalized fading channels with additive Laplacian noise , 2014, 2014 IEEE International Symposium on Information Theory.

[20]  Ibrahim C. Abou-Faycal,et al.  A cauchy input achieves the capacity of a Cauchy channel under a logarithmic constraint , 2014, 2014 IEEE International Symposium on Information Theory.

[21]  Geoffrey Ye Li,et al.  Power of Deep Learning for Channel Estimation and Signal Detection in OFDM Systems , 2017, IEEE Wireless Communications Letters.

[22]  Joohyung Lee,et al.  Deep Learning Based Pilot Allocation Scheme (DL-PAS) for 5G Massive MIMO System , 2018, IEEE Communications Letters.

[23]  Thomas Steinke,et al.  Calibrating Noise to Variance in Adaptive Data Analysis , 2017, COLT.

[24]  Geoffrey Ye Li,et al.  ComNet: Combination of Deep Learning and Expert Knowledge in OFDM Receivers , 2018, IEEE Communications Letters.

[25]  Xuetian Wang,et al.  Deep Neural Networks for Channel Estimation in Underwater Acoustic OFDM Systems , 2019, IEEE Access.

[26]  Biing-Hwang Juang,et al.  Deep Learning in Physical Layer Communications , 2018, IEEE Wireless Communications.

[27]  Vishnu Raj,et al.  Backpropagating Through the Air: Deep Learning at Physical Layer Without Channel Models , 2018, IEEE Communications Letters.

[28]  Jeffrey G. Andrews,et al.  Statistics of Co-Channel Interference in a Field of Poisson and Poisson-Poisson Clustered Interferers , 2010, IEEE Transactions on Signal Processing.

[29]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[30]  Jeffrey G. Andrews,et al.  One-Bit OFDM Receivers via Deep Learning , 2018, IEEE Transactions on Communications.

[31]  Jakob Hoydis,et al.  Model-Free Training of End-to-End Communication Systems , 2018, IEEE Journal on Selected Areas in Communications.

[32]  Stephan ten Brink,et al.  OFDM-Autoencoder for End-to-End Learning of Communications Systems , 2018, 2018 IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[33]  Pascal Fua,et al.  Imposing Hard Constraints on Deep Networks: Promises and Limitations , 2017, CVPR 2017.

[34]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[35]  Jakob Hoydis,et al.  An Introduction to Deep Learning for the Physical Layer , 2017, IEEE Transactions on Cognitive Communications and Networking.

[36]  Kiran Karra,et al.  Learning to communicate: Channel auto-encoders, domain specific regularizers, and attention , 2016, 2016 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT).