Distributed Cooperation Under Uncertainty in Drone-Based Wireless Networks: A Bayesian Coalitional Game

We study the resource sharing problem in a drone-based wireless network. We consider a distributed control setting under uncertainty (i.e. unavailability of full information). In particular, the drones cooperate in serving the users while pooling their spectrum and energy resources in the absence of prior knowledge about different system characteristics such as the amount of available power at the other drones. We cast the aforementioned problem as a Bayesian cooperative game in which the agents (drones) engage in a coalition formation process, where the goal is to maximize the overall transmission rate of the network. The drones update their beliefs using a novel technique that combines the maximum likelihood estimation with Kullback-Leibler divergence. We propose a decision-making strategy for repeated coalition formation that converges to a stable coalition structure. We analyze the performance of the proposed approach by both theoretical analysis and simulations.

[1]  Ekram Hossain,et al.  Multi-Tier Drone Architecture for 5G/B5G Cellular Networks: Challenges, Trends, and Prospects , 2017, IEEE Communications Magazine.

[2]  Mehdi Bennis,et al.  UAV-Assisted Heterogeneous Networks for Capacity Enhancement , 2016, IEEE Communications Letters.

[3]  D. A. Bell,et al.  Information Theory and Reliable Communication , 1969 .

[4]  Craig Boutilier,et al.  Bayesian reinforcement learning for coalition formation under uncertainty , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[5]  Alia Asheralieva,et al.  Bayesian Reinforcement Learning-Based Coalition Formation for Distributed Resource Sharing by Device-to-Device Users in Heterogeneous Cellular Networks , 2017, IEEE Transactions on Wireless Communications.

[6]  Dusit Niyato,et al.  Coalition-Based Cooperative Packet Delivery under Uncertainty: A Dynamic Bayesian Coalitional Game , 2013, IEEE Transactions on Mobile Computing.

[7]  Vikram Krishnamurthy,et al.  Coalition Formation for Bearings-Only Localization in Sensor Networks—A Cooperative Game Approach , 2010, IEEE Transactions on Signal Processing.

[8]  Ying-Chang Liang,et al.  Joint Uplink and Downlink 3D Optimization of an UAV Swarm for Wireless-Powered NB-IoT , 2019, 2019 IEEE Global Communications Conference (GLOBECOM).

[9]  Janne J. Lehtomäki,et al.  Modeling the Dynamics of Coalition Formation Games for Cooperative Spectrum Sharing in an Interference Channel , 2011, IEEE Transactions on Computational Intelligence and AI in Games.

[10]  Chadi Assi,et al.  Joint Location and Beamforming Design for Cooperative UAVs With Limited Storage Capacity , 2019, IEEE Transactions on Communications.

[11]  Sundeep Rangan,et al.  Towards 6G Networks: Use Cases and Technologies , 2019, ArXiv.

[12]  Ali Ghrayeb,et al.  Trajectory Planning and Resource Allocation of Multiple UAVs for Data Delivery in Vehicular Networks , 2019, IEEE Networking Letters.

[13]  Yueming Cai,et al.  A Coalition Formation Framework for Transmission Scheme Selection in Wireless Sensor Networks , 2011, IEEE Transactions on Vehicular Technology.

[14]  Sofie Pollin,et al.  Optimal UAV Positioning for Terrestrial-Aerial Communication in Presence of Fading , 2016, 2016 IEEE Global Communications Conference (GLOBECOM).

[15]  Walid Saad,et al.  Author manuscript, published in "IEEE Transactions on Wireless Communications (2009) Saad-ITransW-2009" A Distributed Coalition Formation Framework for Fair User Cooperation in Wireless Networks , 2022 .

[16]  Setareh Maghsudi,et al.  Distributed User Association in Energy Harvesting Small Cell Networks: An Exchange Economy With Uncertainty , 2017, IEEE Transactions on Green Communications and Networking.

[17]  Hakim Ghazzai,et al.  Trajectory Optimization for Cooperative Dual-Band UAV Swarms , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[18]  Catherine Rosenberg,et al.  Joint Resource Allocation and User Association for Heterogeneous Wireless Cellular Networks , 2013, IEEE Transactions on Wireless Communications.

[19]  Arumugam Nallanathan,et al.  Multi-Agent Reinforcement Learning-Based Resource Allocation for UAV Networks , 2018, IEEE Transactions on Wireless Communications.

[20]  Walid Saad,et al.  Machine Learning for Predictive On-Demand Deployment of Uavs for Wireless Communications , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[21]  Vikram Krishnamurthy,et al.  A Distributed Coalition Game Approach to Femto-Cloud Formation , 2019, IEEE Transactions on Cloud Computing.

[22]  Zhiguo Ding,et al.  Coalition Formation Approaches for Cooperative Networks With SWIPT , 2017, IEEE Access.

[23]  Walid Saad,et al.  Wireless Communication Using Unmanned Aerial Vehicles (UAVs): Optimal Transport Theory for Hover Time Optimization , 2017, IEEE Transactions on Wireless Communications.

[24]  Setareh Maghsudi,et al.  Distributed User Association in Energy Harvesting Small Cell Networks: A Probabilistic Bandit Model , 2016, IEEE Transactions on Wireless Communications.

[25]  Yueming Cai,et al.  A Cooperative Communication Scheme Based on Coalition Formation Game in Clustered Wireless Sensor Networks , 2012, IEEE Transactions on Wireless Communications.

[26]  Walid Saad,et al.  Liquid State Machine Learning for Resource Allocation in a Network of Cache-Enabled LTE-U UAVs , 2017, GLOBECOM 2017 - 2017 IEEE Global Communications Conference.

[27]  Nirwan Ansari,et al.  A Framework for 5G Networks with In-Band Full-Duplex Enabled Drone-Mounted Base-Stations , 2019, IEEE Wireless Communications.

[28]  Halim Yanikomeroglu,et al.  Backhaul-aware robust 3D drone placement in 5G+ wireless networks , 2017, 2017 IEEE International Conference on Communications Workshops (ICC Workshops).

[29]  Qingqing Wu,et al.  Joint Optimization of User Association, Subchannel Allocation, and Power Allocation in Multi-Cell Multi-Association OFDMA Heterogeneous Networks , 2017, IEEE Transactions on Communications.

[30]  Setareh Maghsudi,et al.  Distributed Task Management in Cyber-Physical Systems: How to Cooperate Under Uncertainty? , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[31]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[32]  Abbas Jamalipour,et al.  Modeling air-to-ground path loss for low altitude platforms in urban environments , 2014, 2014 IEEE Global Communications Conference.

[33]  Craig Boutilier,et al.  Sequentially optimal repeated coalition formation under uncertainty , 2012, Autonomous Agents and Multi-Agent Systems.

[34]  Ulrich Schwalbe,et al.  Dynamic Coalition Formation and the Core , 2002 .

[35]  Zhen Zhang,et al.  Wireless Power Transfer—An Overview , 2019, IEEE Transactions on Industrial Electronics.

[36]  Rui Zhang,et al.  Placement Optimization of UAV-Mounted Mobile Base Stations , 2016, IEEE Communications Letters.

[37]  Halim Yanikomeroglu,et al.  On the Number and 3D Placement of Drone Base Stations in Wireless Cellular Networks , 2016, 2016 IEEE 84th Vehicular Technology Conference (VTC-Fall).

[38]  Fei Hu,et al.  Deep ${Q}$ -Learning-Based Node Positioning for Throughput-Optimal Communications in Dynamic UAV Swarm Network , 2019, IEEE Transactions on Cognitive Communications and Networking.

[39]  Dario Pompili,et al.  Cooperative Hierarchical Caching in 5G Cloud Radio Access Networks , 2017, IEEE Network.

[40]  Yiyang Pei,et al.  Deep Reinforcement Learning for User Association and Resource Allocation in Heterogeneous Cellular Networks , 2019, IEEE Transactions on Wireless Communications.

[41]  Halim Yanikomeroglu,et al.  Strategic Densification With UAV-BSs in Cellular Networks , 2018, IEEE Wireless Communications Letters.

[42]  Ekram Hossain,et al.  Statistical Performance Modeling of Solar and Wind-Powered UAV Communications , 2021, IEEE Transactions on Mobile Computing.

[43]  Zhu Han,et al.  A Bayesian Overlapping Coalition Formation Game for Device-to-Device Spectrum Sharing in Cellular Networks , 2015, IEEE Transactions on Wireless Communications.

[44]  Rui Zhang,et al.  Wireless communications with unmanned aerial vehicles: opportunities and challenges , 2016, IEEE Communications Magazine.

[45]  Kemal Leblebicioglu,et al.  Optimal energy allocation policies for a high altitude flying wireless access point , 2015, Trans. Emerg. Telecommun. Technol..