Contention Window Optimization in IEEE 802.11ax Networks with Deep Reinforcement Learning

The proper setting of contention window (CW) values has a significant impact on the efficiency of Wi-Fi networks. Unfortunately, the standard method used by 802.11 networks is not scalable enough to maintain stable throughput for an increasing number of stations, despite 802.11ax being designed to improve Wi-Fi performance in dense scenarios. To this end we propose a new method of CW control which leverages deep reinforcement learning (DRL) principles to learn the correct settings under different network conditions. Our method, called CCOD (Centralized Contention window Optimization with DRL), supports two trainable control algorithms, which, as we demonstrate through simulations, offer efficiency close to optimal while keeping computational cost low.

[1]  Byung-Seo Kim,et al.  Deep Reinforcement Learning Paradigm for Performance Optimization of Channel Observation–Based MAC Protocols in Dense WLANs , 2019, IEEE Access.

[2]  Szymon Szott,et al.  CADWAN: A Control Architecture for Dense WiFi Access Networks , 2018, IEEE Communications Magazine.

[3]  Jun Liu,et al.  GPLM: An 802.11ac-Capable Low-MAC Architecture for FPGA-based SDR Systems , 2019, 2019 IEEE Wireless Communications and Networking Conference (WCNC).

[4]  Anatolij Zubow,et al.  ns-3 meets OpenAI Gym: The Playground for Machine Learning in Networking Research , 2019, MSWiM.

[5]  Luliang Jia,et al.  A Collaborative Multi-Agent Reinforcement Learning Anti-Jamming Algorithm in Wireless Networks , 2018, IEEE Wireless Communications Letters.

[6]  Cristina Cano,et al.  A Flexible Machine-Learning-Aware Architecture for Future WLANs , 2020, IEEE Communications Magazine.

[7]  F. Al-Tam,et al.  Learn to Schedule (LEASCH): A Deep Reinforcement Learning Approach for Radio Resource Scheduling in the 5G MAC Layer , 2020, IEEE Access.

[8]  Thomas Sandholm,et al.  Learning to Wait: Wi-Fi Contention Control using Load-based Predictions , 2019, ArXiv.

[9]  Guy Lever,et al.  Deterministic Policy Gradient Algorithms , 2014, ICML.

[10]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[11]  Boris Bellalta,et al.  AP-initiated Multi-User Transmissions in IEEE 802.11ax WLANs , 2017, Ad Hoc Networks.

[12]  Ronghong Mo,et al.  Reconfigurable Intelligent Surface Assisted Multiuser MISO Systems Exploiting Deep Reinforcement Learning , 2020, IEEE Journal on Selected Areas in Communications.

[13]  Chunxiao Jiang,et al.  Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks , 2019, IEEE Communications Surveys & Tutorials.

[14]  Hamed Haddadi,et al.  Deep Learning in Mobile and Wireless Networking: A Survey , 2018, IEEE Communications Surveys & Tutorials.

[15]  Soung Chang Liew,et al.  Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks , 2017, 2018 IEEE International Conference on Communications (ICC).

[16]  Mehmet Karaca,et al.  Modifying Backoff Freezing Mechanism to Optimize Dense IEEE 802.11 Networks , 2017, IEEE Transactions on Vehicular Technology.

[17]  A. Girotra,et al.  Performance Analysis of the IEEE 802 . 11 Distributed Coordination Function , 2005 .

[18]  Liqun Fu,et al.  Deep Reinforcement Learning Based MAC Protocol for Underwater Acoustic Networks , 2019, IEEE Transactions on Mobile Computing.

[19]  Evgeny Khorov,et al.  A Tutorial on IEEE 802.11ax High Efficiency WLANs , 2019, IEEE Communications Surveys & Tutorials.

[20]  Vincenzo Mancuso,et al.  Control theoretic optimization of 802.11 WLANs: Implementation and experimental evaluation , 2012, Comput. Networks.

[21]  Ramesh Raskar,et al.  Distributed learning of deep neural network over multiple agents , 2018, J. Netw. Comput. Appl..