Reinforcement Learning Based Load Balancing for Hybrid LiFi WiFi Networks

Light fidelity (LiFi) is an emerging communication technology, which utilizes the light-emitting diodes (LEDs) for high-speed wireless communications. Due to its huge unlicensed bandwidth, LiFi is capable of supporting high data rates. The quality of the LiFi channel fluctuates across the room due to interference, reflection from walls or blockage. On the other hand, WiFi is another wireless communication technology that is capable of providing moderate data rates with ubiquitous coverage. As the electromagnetic spectrum of LiFi does not overlap with WiFi, both of them can coexist to form a hybrid LiFi and WiFi network for seamless and high-throughput connectivity. The performance of a hybrid system significantly depends upon the access point (AP) assignment and resource allocation strategies. In this paper, a downlink hybrid system with one WiFi AP and four LiFi APs is considered, and a reinforcement learning (RL) algorithm is implemented in order to determine an optimal AP assignment strategy, which maximizes the long-term system throughput while ensuring the required users fairness and satisfaction. Furthermore, two different scenarios based on the random waypoint model with uniform and non-uniform distribution of users have been studied. The performance of the proposed system is compared against state-of-the-art benchmark approaches e.g., signal strength strategy (SSS), exhaustive search, and an iterative optimization method. The results are reported in terms of the average system throughput, user satisfaction, fairness, and capacity outage probability. It is shown that the proposed RL method performs closer to the exhaustive search scheme at fairly low complexity. The RL method also outperforms the SSS scheme and the iterative algorithm in most scenarios.

[1]  Murat Uysal,et al.  Unified Resource Allocation and Mobility Management Technique Using Particle Swarm Optimization for VLC Networks , 2018, IEEE Photonics Journal.

[2]  Harald Haas,et al.  Access point assignment in hybrid LiFi and WiFi networks in consideration of LiFi channel blockage , 2017, 2017 IEEE 18th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[3]  Lin Zhang,et al.  QoE-Driven Optimized Load Balancing Design for Hybrid LiFi and WiFi Networks , 2018, IEEE Communications Letters.

[4]  Stefan Videv,et al.  Dynamic load balancing with handover in hybrid Li-Fi and Wi-Fi networks , 2014, 2014 IEEE 25th Annual International Symposium on Personal, Indoor, and Mobile Radio Communication (PIMRC).

[5]  Dale Schuurmans,et al.  Trust-PCL: An Off-Policy Trust Region Method for Continuous Control , 2017, ICLR.

[6]  R. Bellman A Markovian Decision Process , 1957 .

[7]  Mostafa Zaman Chowdhury,et al.  Optical Wireless Hybrid Networks: Trends, Opportunities, Challenges, and Research Directions , 2020, IEEE Communications Surveys & Tutorials.

[8]  Zhiyong Du,et al.  Reinforcement Learning Based Network Selection for Hybrid VLC and RF Systems , 2018 .

[9]  Anna Maria Vegni,et al.  A hybrid Radio Frequency and broadcast Visible Light Communication system , 2011, 2011 IEEE GLOBECOM Workshops (GC Wkshps).

[10]  Henrik Schulze,et al.  Frequency-Domain Simulation of the Indoor Wireless Optical Communication Channel , 2016, IEEE Transactions on Communications.

[11]  Harald Haas,et al.  LiFi: Conceptions, misconceptions and opportunities , 2016, 2016 IEEE Photonics Conference (IPC).

[12]  Liang Yin,et al.  Introduction to indoor networking concepts and challenges in LiFi , 2019, IEEE/OSA Journal of Optical Communications and Networking.

[13]  Parth H. Pathak,et al.  Visible Light Communication, Networking, and Sensing: A Survey, Potential and Challenges , 2015, IEEE Communications Surveys & Tutorials.

[14]  Junyi Li,et al.  Visible light communication: opportunities, challenges and the path to market , 2013, IEEE Communications Magazine.

[15]  Gang Niu,et al.  Analysis and Improvement of Policy Gradient Estimation , 2011, NIPS.

[16]  Mohammad Dehghani Soltani,et al.  Hybrid LiFi and WiFi Networks: A Survey , 2020, IEEE Communications Surveys & Tutorials.

[17]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[18]  Christian Wagner,et al.  The Spatial Node Distribution of the Random Waypoint Mobility Model , 2002, WMAN.

[19]  Harald Haas,et al.  Optimization of Load Balancing in Hybrid LiFi/RF Networks , 2017, IEEE Transactions on Communications.

[20]  Lajos Hanzo,et al.  Anticipatory Association for Indoor Visible Light Communications: Light, Follow Me! , 2018, IEEE Transactions on Wireless Communications.

[21]  Pieter Abbeel,et al.  Model-Ensemble Trust-Region Policy Optimization , 2018, ICLR.

[22]  Harald Haas,et al.  Access Point Selection for Hybrid Li-Fi and Wi-Fi Networks , 2017, IEEE Transactions on Communications.

[23]  Yunhao Tang,et al.  Boosting Trust Region Policy Optimization by Normalizing Flows Policy , 2018, ArXiv.

[24]  Harald Haas,et al.  Bidirectional User Throughput Maximization Based on Feedback Reduction in LiFi Networks , 2017, IEEE Transactions on Communications.

[25]  Harald Haas,et al.  Joint Optimisation of Load Balancing and Handover for Hybrid LiFi and WiFi Networks , 2017, 2017 IEEE Wireless Communications and Networking Conference (WCNC).

[26]  Li Chen,et al.  Design and realization of indoor VLC-Wi-Fi hybrid network , 2017, Journal of Communications and Information Networks.

[27]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[28]  Volker Jungnickel,et al.  High-speed visible light communication systems , 2013, IEEE Communications Magazine.

[29]  Harald Haas,et al.  Fuzzy logic based dynamic handover scheme for indoor Li-Fi and RF hybrid network , 2016, 2016 IEEE International Conference on Communications (ICC).

[30]  Jiaheng Wang,et al.  Beam Domain Massive MIMO for Optical Wireless Communications With Transmit Lens , 2017, IEEE Transactions on Communications.

[31]  Victor C. M. Leung,et al.  Learning-Aided Network Association for Hybrid Indoor LiFi-WiFi Systems , 2018, IEEE Transactions on Vehicular Technology.

[32]  H. Haas,et al.  A 3-Gb/s Single-LED OFDM-Based Wireless VLC Link Using a Gallium Nitride $\mu{\rm LED}$ , 2014, IEEE Photonics Technology Letters.

[33]  Mohamed M. Abdallah,et al.  Energy Efficient Resource Allocation for Mixed RF/VLC Heterogeneous Wireless Networks , 2016, IEEE Journal on Selected Areas in Communications.

[34]  Mostafa Zaman Chowdhury,et al.  A Comparative Survey of Optical Wireless Technologies: Architectures and Applications , 2018, IEEE Access.

[35]  Harald Haas,et al.  A guide to wireless networking by light , 2017 .

[36]  H. Haas,et al.  LED Based Wavelength Division Multiplexed 10 Gb/s Visible Light Communications , 2016, Journal of Lightwave Technology.

[37]  George K. Karagiannidis,et al.  Li-Fi and Wi-Fi with common backhaul: Coordination and resource allocation , 2018, 2018 IEEE Wireless Communications and Networking Conference (WCNC).

[38]  Hang Li,et al.  Achievable Rate With Closed-Form for SISO Channel and Broadcast Channel in Visible Light Communication Networks , 2017, Journal of Lightwave Technology.

[39]  Volker Jungnickel,et al.  Coexistence of WiFi and LiFi toward 5G: concepts, opportunities, and challenges , 2016, IEEE Communications Magazine.