DM-CSAT: a LTE-U/Wi-Fi coexistence solution based on reinforcement learning

Recent literature demonstrated promising results of Long-Term Evolution (LTE) deployments over unlicensed bands when coexisting with Wi-Fi networks via the Duty-Cycle (DC) approach. However, it is known that performance in coexistence is strongly dependent on traffic patterns and on the duty-cycle ON–OFF rate of LTE. Most DC solutions rely on static coexistence parameters configuration, hence real-life performance in dynamically varying scenarios might be affected. Advanced reinforcement learning techniques may be used to adjust DC parameters towards efficient coexistence, and we propose a Q-learning Carrier-Sensing Adaptive Transmission mechanism which adapts LTE duty-cycle ON–OFF time ratio to the transmitted data rate, aiming at maximizing the Wi-Fi and LTE-Unlicensed (LTE-U) aggregated throughput. The problem is formulated as a Markov decision process, and the Q-learning solution for finding the best LTE-U ON–OFF time ratio is based on the Bellman’s equation. We evaluate the performance of the proposed solution for different traffic load scenarios using the ns-3 simulator. Results demonstrate the benefits from the adaptability to changing circumstances of the proposed method in terms of Wi-Fi/LTE aggregated throughput, as well as achieving a fair coexistence.

[1]  Ismail Güvenç,et al.  Reinforcement learning for licensed-assisted access of LTE in the unlicensed spectrum , 2015, 2015 IEEE Wireless Communications and Networking Conference (WCNC).

[2]  Cristina Cano,et al.  Fair Coexistence of Scheduled and Random Access Wireless Networks: Unlicensed LTE/WiFi , 2016, IEEE/ACM Transactions on Networking.

[3]  Walid Saad,et al.  Echo State Networks for Self-Organizing Resource Allocation in LTE-U With Uplink–Downlink Decoupling , 2016, IEEE Transactions on Wireless Communications.

[4]  Eugene A. Feinberg,et al.  Handbook of Markov Decision Processes , 2002 .

[5]  Yimin Pang,et al.  Wi-Fi Coexistence with Duty Cycled LTE-U , 2016, Wirel. Commun. Mob. Comput..

[6]  Pedro Maia De Santana,et al.  Performance of License Assisted Access Solutions Using ns-3 , 2016, 2016 International Conference on Computational Science and Computational Intelligence (CSCI).

[7]  Sayantan Choudhury,et al.  Enabling the coexistence of LTE and Wi-Fi in unlicensed bands , 2014, IEEE Communications Magazine.

[8]  Oriol Sallent,et al.  On the implementation of channel selection for LTE in unlicensed bands using Q-learning and Game Theory algorithms , 2017, 2017 13th International Wireless Communications and Mobile Computing Conference (IWCMC).

[9]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[10]  R. Kwan,et al.  Fair co-existence of Licensed Assisted Access LTE (LAA-LTE) and Wi-Fi in unlicensed spectrum , 2015, 2015 7th Computer Science and Electronic Engineering Conference (CEEC).

[11]  Cristina Cano,et al.  Coexistence of WiFi and LTE in unlicensed bands: A proportional fair allocation scheme , 2015, 2015 IEEE International Conference on Communication Workshop (ICCW).

[12]  Jie Zhang,et al.  Coexistence of LTE-LAA and Wi-Fi on 5 GHz With Corresponding Deployment Scenarios: A Survey , 2017, IEEE Communications Surveys & Tutorials.

[13]  Reid G. Simmons,et al.  Complexity Analysis of Real-Time Reinforcement Learning , 1993, AAAI.

[14]  Amr El-Helw,et al.  Performance Evaluation of MAC for IEEE 802.11 and LAA LTE , 2016, 2016 International Conference on Computational Science and Computational Intelligence (CSCI).

[15]  Xiaohu You,et al.  LTE on License-Exempt Spectrum , 2018, IEEE Communications Surveys & Tutorials.

[16]  Xiaoli Chu,et al.  Performance Analysis of LAA and WiFi Coexistence in Unlicensed Spectrum Based on Markov Chain , 2016, 2016 IEEE Global Communications Conference (GLOBECOM).

[17]  Sayantan Choudhury,et al.  Enabling LTE/WiFi coexistence by LTE blank subframe allocation , 2013, 2013 IEEE International Conference on Communications (ICC).

[18]  D. Stroock.  An Introduction to Markov Processes , 2004 .

[19]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[20]  R. Bellman.  Dynamic programming. , 1957, Science.

[21]  Naceur Malouch,et al.  Studying the impact of LTE-U on Wi-Fi downlink performance , 2016, 2016 IEEE 12th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob).

[22]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[23]  Fotis Foukalas,et al.  Efficient Coexistence of LTE With WiFi in the Licensed and Unlicensed Spectrum Aggregation , 2016, IEEE Transactions on Cognitive Communications and Networking.

[24]  Min Sheng,et al.  Throughput and Fairness Analysis of Wi-Fi and LTE-U in Unlicensed Band , 2017, IEEE Journal on Selected Areas in Communications.

[25]  Kiran Kuchi,et al.  Performance Evaluation of LAA-LBT Based LTE and WLAN's Co-Existence in Unlicensed Spectrum , 2015, 2015 IEEE Globecom Workshops (GC Wkshps).

[26]  Dries Naudts,et al.  An adaptive LTE listen-before-talk scheme towards a fair coexistence with Wi-Fi in unlicensed spectrum , 2018, Telecommunication Systems.

[27]  Nadisanka Rupasinghe,et al.  Licensed-assisted access for WiFi-LTE coexistence in the unlicensed spectrum , 2014, 2014 IEEE Globecom Workshops (GC Wkshps).

[28]  Yimin Pang,et al.  On the Impact of LTE-U on Wi-Fi Performance , 2014, 2014 IEEE 25th Annual International Symposium on Personal, Indoor, and Mobile Radio Communication (PIMRC).