Graph Attention Network-based DRL for Network Slicing Management in Dense Cellular Networks

Network slicing (NS) devotes to provisioning various services with distinct requirements over the same physical communication infrastructure. Considering a dense cellular network scenario that contains several NS over multiple base stations (BSs), it remains challenging to design a proper resource management strategy in real time, so as to cope with frequent BS handover and meet distinct service requirements. In this paper, we propose to formulate this challenge as a multiagent reinforcement learning (MARL) problem and leverage graph attention network (GAT) to strengthen the cooperation between agents. Furthermore, we incorporate GAT into deep Q network (DQN) and correspondingly design an intelligent resource management strategy for NS. Finally, we verify the superiority of the GAT-based DQN algorithm through extensive simulations.

[1]  Zongqing Lu,et al.  Graph Convolutional Reinforcement Learning for Multi-Agent Cooperation , 2018, ArXiv.

[2]  Xianfu Chen,et al.  GAN-Based Deep Distributional Reinforcement Learning for Resource Management in Network Slicing , 2019, 2019 IEEE Global Communications Conference (GLOBECOM).

[3]  Bin Han,et al.  Slice as an Evolutionary Service: Genetic Optimization for Inter-Slice Resource Management in 5G Networks , 2018, IEEE Access.

[4]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[5]  Tom Schaul,et al.  Dueling Network Architectures for Deep Reinforcement Learning , 2015, ICML.

[6]  Gunnar Mildh,et al.  Impact of network slicing on 5G Radio Access Networks , 2016, 2016 European Conference on Networks and Communications (EuCNC).

[7]  Nan Xu,et al.  CoLight: Learning Network-level Cooperation for Traffic Signal Control , 2019, CIKM.

[8]  Zhifeng Zhao,et al.  The LSTM-Based Advantage Actor-Critic Learning for Resource Management in Network Slicing With User Mobility , 2020, IEEE Communications Letters.

[9]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[10]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[11]  Yan Chen,et al.  Intelligent 5G: When Cellular Networks Meet Artificial Intelligence , 2017, IEEE Wireless Communications.

[12]  Lei Zhang,et al.  User Access Control and Bandwidth Allocation for Slice-Based 5G-and-Beyond Radio Access Networks , 2019, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[13]  Xianfu Chen,et al.  Deep Reinforcement Learning for Resource Management in Network Slicing , 2018, IEEE Access.

[14]  Tom Schaul,et al.  Rainbow: Combining Improvements in Deep Reinforcement Learning , 2017, AAAI.

[15]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.