Deep Reinforcement Learning-Based Channel Allocation for Wireless LANs with Graph Convolutional Networks

To enhance the network throughput of densely deployed wireless local area networks (WLANs), coordinated control methods of access points (APs) have recently been discussed, especially within the IEEE 802.11 Extremely High Throughput Study Group. This paper presents a deep reinforcement learning-based channel allocation scheme using graph convolutional networks (GCNs), as a coordinated control method of APs. In densely deployed WLANs, the number of observable topologies of APs is extremely high, and thus, we extract the features of the topological structures based on GCNs. We apply GCNs to a contention graph where APs within their carrier sensing ranges are connected to extract the features of carrier sensing relationships. Additionally, to improve the learning speed, especially in an early stage of learning, we employ a game theory-based method to collect training data, independent of the neural network model. The simulation results indicate that the proposed method can allocate channels with higher throughput when compared to existing methods.

[1]  Joan Bruna,et al.  Deep Convolutional Networks on Graph-Structured Data , 2015, ArXiv.

[2]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[3]  L. Shapley,et al.  Potential Games , 1994 .

[4]  George Karypis,et al.  Frequent subgraph discovery , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[5]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[6]  Tom Schaul,et al.  Dueling Network Architectures for Deep Reinforcement Learning , 2015, ICML.

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Robert Babuska,et al.  Experience Replay for Real-Time Reinforcement Learning Control , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[9]  Theodore S. Rappaport,et al.  Wireless communications - principles and practice , 1996 .

[10]  Brendan D. McKay,et al.  Practical graph isomorphism, II , 2013, J. Symb. Comput..

[11]  Koji Yamamoto,et al.  A Comprehensive Survey of Potential Game Approaches to Wireless Networks , 2015, IEICE Trans. Commun..

[12]  Petteri Kaski,et al.  Engineering an Efficient Canonical Labeling Tool for Large and Sparse Graphs , 2007, ALENEX.

[13]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[14]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[15]  H. Young,et al.  Individual Strategy and Social Structure: An Evolutionary Theory of Institutions , 1999 .

[16]  Soung Chang Liew,et al.  Back-of-the-Envelope Computation of Throughput Distributions in CSMA Wireless Networks , 2007, 2009 IEEE International Conference on Communications.

[17]  Tom Schaul,et al.  Prioritized Experience Replay , 2015, ICLR.