Optimal Channel Selection in Hybrid RF/VLC Networks: A Multi-Armed Bandit Approach

We investigate optimal band/channel selection in hybrid radio frequency and visible light communication (RF/VLC) networks. Particularly, we first develop a robust hybrid RF/VLC based system model for the optimal band/channel selection. We then formulate it as an online stochastic budget-constrained multi-armed bandit (MAB) problem. Two online learning algorithms based on different optimal policies are proposed to choose the appropriate band, i.e., energy-aware band selection with upper confidence bound (EABS-UCB) and energy-aware band selection with Thompson sampling (EABS-TS). The cost/budget is the battery consumption of the transmitting device according to the selected band. Through extensive simulations, it is confirmed that the proposed EABS-TS emerges as the superior technique compared with the random, brute-force, and EABS-UCB band selection schemes, in terms of energy efficiency, average throughput, and convergence performance.

[1]  S. Ikki,et al.  Allocation Schemes for Relay Communications: A Multiband Multichannel Approach Using Game Theory , 2022, IEEE Sensors Letters.

[2]  Ehab Mahmoud Mohamed,et al.  Two-Hop Relay Probing in WiGig Device-to-Device Networks Using Sleeping Contextual Bandits , 2021, IEEE Wireless Communications Letters.

[3]  Feng Lyu,et al.  Multi-Path Selection and Congestion Control for NDN: An Online Learning Approach , 2021, IEEE Transactions on Network and Service Management.

[4]  Ehab Mahmoud Mohamed,et al.  Wi-Fi Assisted Contextual Multi-Armed Bandit for Neighbor Discovery and Selection in Millimeter Wave Device to Device Communications , 2021, Sensors.

[5]  Nidal Nasser,et al.  A Deep Learning Method for Predictive Channel Assignment in Beyond 5G Networks , 2021, IEEE Network.

[6]  Nei Kato,et al.  AI Models for Green Communications Towards 6G , 2021, IEEE Communications Surveys & Tutorials.

[7]  E. M. Mohamed,et al.  Multiagent Multi-Armed Bandit Techniques for Millimeter Wave Concurrent Beamforming , 2020, 2020 8th International Japan-Africa Conference on Electronics, Communications, and Computations (JAC-ECC).

[8]  E. M. Mohamed,et al.  Multiagent Multi-Armed Bandit Schemes for Gateway Selection in UAV Networks , 2020, 2020 IEEE Globecom Workshops (GC Wkshps.

[9]  Sabit Ekin,et al.  Hybrid RF/VLC Systems: A Comprehensive Survey on Network Topologies, Performance Analyses, Applications, and Future Directions , 2020, IEEE Access.

[10]  Zdenek Becvar,et al.  Deep Learning for Selection Between RF and VLC Bands in Device-to-Device Communication , 2020, IEEE Wireless Communications Letters.

[11]  Qing Zhao,et al.  Multi-Armed Bandits: Theory and Applications to Online Learning in Networks , 2019, Multi-Armed Bandits.

[12]  Ning Zhang,et al.  Fast mmwave Beam Alignment via Correlated Bandit Learning , 2019, IEEE Transactions on Wireless Communications.

[13]  Zhu Han,et al.  Resource Allocation for Device-to-Device Communications in Multi-Cell Multi-Band Heterogeneous Cellular Networks , 2019, IEEE Transactions on Vehicular Technology.

[14]  Nei Kato,et al.  Multi-Hop Wireless Transmission in Multi-Band WLAN Systems: Proposal and Future Perspective , 2019, IEEE Wireless Communications.

[15]  Christian Schneider,et al.  Multi-Band Vehicle-to-Vehicle Channel Characterization in the Presence of Vehicle Blockage , 2019, IEEE Access.

[16]  Ghaïs El Zein,et al.  Indoor Path Loss Measurements and Modeling in an Open-Space Office at 2.4 GHz and 5.8 GHz in the Presence of People , 2018, 2018 IEEE 29th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC).

[17]  Harald Haas,et al.  Link Selection in Hybrid RF/VLC Systems Under Statistical Queueing Constraints , 2017, IEEE Transactions on Wireless Communications.

[18]  Nei Kato,et al.  Survey on Machine Learning for Intelligent End-to-End Communication Toward 6G: From Network Access, Routing to Traffic Control and Streaming Adaption , 2021, IEEE Communications Surveys & Tutorials.

[19]  Hui Wang,et al.  Deep Q-Network Learning Based Downlink Resource Allocation for Hybrid RF/VLC Systems , 2020, IEEE Access.