Reinforcement Learning for Physical Layer Communications

Wireless communication systems have to be designed in order to cope with time-frequency-space varying channel conditions and variety of interference sources. In cellular wireless systems for instance, channel is estimated regularly by mobile terminals and base stations (BS) using dedicated pilot signals. This allows for adapting the transmitters and receivers to the current channel conditions and interference scenario. Powerful adaptive signal processing algorithms have been developed in the past decades in order to cope with the dynamic nature of the wireless channel, e.g. the least mean square and recursive least square algorithms for channel equalization or estimation, the Kalman filtering in multiple-input multiple-output channel matrix and frequency offset tracking. These techniques rely on very well established mathematical models of physical phenomena that allow to derive the optimal processing for a given criterion, e.g. mean square error and assumed noise and interference distribution models. Any mathematical model trades-off between its complexity and its tractability. A very complete, and hence complex, model may be useless if any insight on the state of the system cannot be drawn easily. For instance, the wireless propagation channel is absolutely deterministic and the signal received at any point of the space at any time can be precisely predicted by the Maxwell equations. However, this would require a prohibitive amount of computation and memory storage for a receiver to calculate at any point the value of the electric and magnetic fields using detailed and explicit knowledge of the physical characteristics of scatterers in the propagation environment, e.g. the dielectric and permittivity constants of the walls and other obstacles. It is much more efficient to design receivers that perform well in environments that have been stochastically characterized instead of using explicit deterministic model of each particular propagation environment. Modern and emerging wireless systems are characterized by massive amounts of connected mobile devices, BS, sensors and actuators. Modeling such large scale wireless systems has become a formidable task because of, for example, very small cell sizes, channel aware link adaptation and waveform deployment, diversity techniques and optimization of the use of different degrees of freedom in tranceivers. Consequently, it may not be feasible to build explicit and detailed mathematical models of wireless systems and their operational environments. In fact, there is a serious modeling deficit that calls for creating awareness of the operational wireless environment through sensing and learning. Machine learning (ML) refers to a large class of algorithms that aim at giving to a machine the capability to acquire knowledge or behavior. If the machine is a wireless system, which is man-made, then