Reinforcement Learning Assisted Beamforming for Inter-cell Interference Mitigation in 5G Massive MIMO Networks

Beamforming is an essential technology in the 5Gmassive multipleinput-multiple-output (MMIMO) communications, which are subject to many impairments due to the nature of wireless transmission channel, i.e. the air. The inter-cell interference (ICI) is one of the main impairments faced by 5G communications due to frequencyreuse technologies. In this paper, we propose a reinforcement learning (RL) assisted full dynamic beamforming for ICI mitigation in 5G downlink. The proposed algorithm is a joint of beamforming and full dynamic Q-learning technology to minimize the ICI, and results in a low-complexity method without channel estimation. Performance analysis shows the quality of service improvement in terms of signal-to-interference-plus-noise-ratio (SINR) and computational complexity compared to other algorithms. CCS CONCEPTS • Networks → Wireless access points, base stations and infrastructure; • Computing methodologies → Model development and analysis; • Theory of computation→Reinforcement learning.

[1]  Anca D. Dragan,et al.  Inverse Reward Design , 2017, NIPS.

[2]  Cyril Leung,et al.  A Survey of Scheduling and Interference Mitigation in LTE , 2010, J. Electr. Comput. Eng..

[3]  Håkan Johansson,et al.  Optimal Channel Estimation for Hybrid Energy Beamforming Under Phase Shifter Impairments , 2019, IEEE Transactions on Communications.

[4]  Ismail Güvenç,et al.  Learning Based Frequency- and Time-Domain Inter-Cell Interference Coordination in HetNets , 2014, IEEE Transactions on Vehicular Technology.

[5]  Florian Kaltenberger,et al.  Practical Hybrid Beamforming Schemes in Massive MIMO 5G NR Systems , 2019, WSA.

[6]  Emil Björnson,et al.  Massive MIMO: ten myths and one critical question , 2015, IEEE Communications Magazine.

[7]  Haralabos C. Papadopoulos,et al.  Hybrid Beamforming With Selection for Multiuser Massive MIMO Systems , 2017, IEEE Transactions on Signal Processing.

[8]  Haitham S. Hamza,et al.  A Survey on Inter-Cell Interference Coordination Techniques in OFDMA-Based Cellular Networks , 2013, IEEE Communications Surveys & Tutorials.

[9]  Guidelines for evaluation of radio interface technologies for IMT-Advanced , 2008 .

[10]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[11]  Mugen Peng,et al.  Reinforcement Learning-Based Interference Control for Ultra-Dense Small Cells , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[12]  Zwi Altman,et al.  A cooperative Reinforcement Learning approach for Inter-Cell Interference Coordination in OFDMA cellular networks , 2010, 8th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks.

[13]  Ganesh K. Venayagamoorthy,et al.  Computational Intelligence in Wireless Sensor Networks: A Survey , 2011, IEEE Communications Surveys & Tutorials.

[14]  Joy Iong-Zong Chen,et al.  Performance evaluation of BER for an Massive-MIMO with M-ary PSK scheme over Three-Dimension correlated channel , 2017, Comput. Electr. Eng..

[15]  Andrew Y. Ng,et al.  Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping , 1999, ICML.

[16]  K. Sandrasegaran,et al.  Survey of intercell interference mitigation techniques in LTE downlink networks , 2012, Australasian Telecommunication Networks and Applications Conference (ATNAC) 2012.

[17]  Klaus I. Pedersen,et al.  Interference coordination for dense wireless networks , 2015, IEEE Communications Magazine.

[18]  Theodore S. Rappaport,et al.  Investigation of Prediction Accuracy, Sensitivity, and Parameter Stability of Large-Scale Propagation Path Loss Models for 5G Wireless Communications , 2016, IEEE Transactions on Vehicular Technology.

[19]  Chen Qian,et al.  Smart Pilot Assignment for Massive MIMO , 2015, IEEE Communications Letters.

[20]  Eric Wiewiora,et al.  Potential-Based Shaping and Q-Value Initialization are Equivalent , 2003, J. Artif. Intell. Res..

[21]  Reid G. Simmons,et al.  Complexity Analysis of Real-Time Reinforcement Learning , 1993, AAAI.

[22]  Vladimir Poulkov,et al.  Combined power and inter-cell interference control for LTE based on role game approach , 2011, 2011 34th International Conference on Telecommunications and Signal Processing (TSP).

[23]  Joohyung Lee,et al.  Deep Learning Based Pilot Allocation Scheme (DL-PAS) for 5G Massive MIMO System , 2018, IEEE Communications Letters.

[24]  Mérouane Debbah,et al.  Massive MIMO in the UL/DL of Cellular Networks: How Many Antennas Do We Need? , 2013, IEEE Journal on Selected Areas in Communications.