Power Allocation in Multi-Cell Networks Using Deep Reinforcement Learning

In this paper, multi-cell power allocation approach is researched. Different from the traditional optimization decomposition method, Deep Reinforcement Learning (DRL) method is employed to solve the power allocation issue which is an NP-hard problem. The objective of our work is to maximize the overall capacity of the entire network in the scenario where the base stations are randomly and densely distributed. We propose a wireless resource mapping method and a deep neural network for multi-cell power allocation named as Deep-Q-Full-Connected-Network (DQFCNet). Compared with the water-filling power allocation and Q-learning method, DQFCNet can achieve a higher overall capacity. Furthermore, the simulation results show that DQFCNet has significant improvement in convergence speed and stability.

[1]  Honghai Zhang,et al.  Weighted Sum-Rate Maximization in Multi-Cell Networks via Coordinated Scheduling and Discrete Power Control , 2011, IEEE Journal on Selected Areas in Communications.

[2]  Tamer A. ElBatt,et al.  A cooperative Q-learning approach for distributed resource allocation in multi-user femtocell networks , 2014, 2014 IEEE Wireless Communications and Networking Conference (WCNC).

[3]  Gang Feng,et al.  Energy-Efficient Joint Scheduling and Power Control in Multi-Cell Wireless Networks , 2016, IEEE Journal on Selected Areas in Communications.

[4]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[5]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[6]  Xiaohu You,et al.  Energy-Efficient Resource Allocation in Coordinated Downlink Multicell OFDMA Systems , 2016, IEEE Transactions on Vehicular Technology.

[7]  Rahim Tafazolli,et al.  Graph-Based Multicell Scheduling in OFDMA-Based Small Cell Networks , 2014, IEEE Access.

[8]  Andreas Mitschele-Thiel,et al.  Multi-parameter Q-Learning for downlink Inter-Cell Interference Coordination in LTE SON , 2014, 2014 IEEE Symposium on Computers and Communications (ISCC).

[9]  Mohamad Yassin,et al.  Cooperative resource management and power allocation for multiuser OFDMA networks , 2017, IET Commun..

[10]  Afef Feki,et al.  Detection of synchronization signals in reuse-1 LTE networks , 2009, 2009 2nd IFIP Wireless Days (WD).

[11]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[12]  Cong Shen,et al.  Learn to adapt: Self-optimizing small cell transmit power with correlated bandit learning , 2017, 2017 IEEE International Conference on Communications (ICC).

[13]  Lie-Liang Yang,et al.  Novel Intercell Interference Mitigation Algorithms for Multicell OFDMA Systems With Limited Base Station Cooperation , 2017, IEEE Transactions on Vehicular Technology.

[14]  Hamed Haddadi,et al.  Deep Learning in Mobile and Wireless Networking: A Survey , 2018, IEEE Communications Surveys & Tutorials.

[15]  Xiaohu You,et al.  Joint Power and Resource Allocation for Non-Uniform Topologies in Heterogeneous Networks , 2016, 2016 IEEE 83rd Vehicular Technology Conference (VTC Spring).

[16]  Ming Chen,et al.  User Association, Resource Allocation and Power Control in Load-Coupled Heterogeneous Networks , 2016, 2016 IEEE Globecom Workshops (GC Wkshps).

[17]  Muhammad Ali Imran,et al.  A Survey of Self Organisation in Future Cellular Networks , 2013, IEEE Communications Surveys & Tutorials.

[18]  Brahim Chaib-draa,et al.  Labeled Initialized Adaptive Play Q-learning for Stochastic Games , 2007 .

[19]  Gang Feng,et al.  Energy-Efficient Downlink Resource Allocation in Heterogeneous OFDMA Networks , 2017, IEEE Transactions on Vehicular Technology.