Deep Reinforcement Learning Autoencoder with Noisy Feedback

End-to-end learning of communication systems enables joint optimization of transmitter and receiver, implemented as deep neural network (NN)-based autoencoders, over any type of channel and for an arbitrary performance metric. Recently, an alternating training procedure was proposed which eliminates the need for an explicit channel model. However, this approach requires feedback of real-valued losses from the receiver to the transmitter during training. In this paper, we first show that alternating training works even with a noisy feedback channel. Then, we design a system that learns to transmit real numbers over an unknown channel without a preexisting feedback link. Once trained, this feedback system can be used to communicate losses during alternating training of autoencoders. Evaluations over additive white Gaussian noise (AWGN) and Rayleigh block-fading (RBF) channels show that end-to-end communication systems trained using the proposed feedback system achieve the same performance as when trained with a perfect feedback link.

[1]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[2]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[3]  Jakob Hoydis,et al.  End-to-End Learning of Communications Systems Without a Channel Model , 2018, 2018 52nd Asilomar Conference on Signals, Systems, and Computers.

[4]  Anant Sahai,et al.  Cooperative Multi-Agent Reinforcement Learning for Low-Level Wireless Communication , 2018, ArXiv.

[5]  Biing-Hwang Juang,et al.  Channel Agnostic End-to-End Learning Based Communication Systems with Conditional GAN , 2018, 2018 IEEE Globecom Workshops (GC Wkshps).

[6]  Vishnu Raj,et al.  Backpropagating Through the Air: Deep Learning at Physical Layer Without Channel Models , 2018, IEEE Communications Letters.

[7]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[8]  Stephan ten Brink,et al.  Deep Learning Based Communication Over the Air , 2017, IEEE Journal of Selected Topics in Signal Processing.

[9]  A. Hall,et al.  Adaptive Switching Circuits , 2016 .

[10]  Timothy J. O'Shea,et al.  Approximating the Void: Learning Stochastic Channel Models from Observation with Variational Generative Adversarial Networks , 2018, 2019 International Conference on Computing, Networking and Communications (ICNC).

[11]  Jakob Hoydis,et al.  An Introduction to Deep Learning for the Physical Layer , 2017, IEEE Transactions on Cognitive Communications and Networking.