Distributed Learning Based Joint Communication and Computation Strategy of IoT Devices in Smart Cities

With the development of global urbanization, the Internet of Things (IoT) and smart cities are becoming hot research topics. As an emerging model, edge computing can play an important role in smart cities because of its low latency and good performance. IoT devices can reduce time consumption with the help of a mobile edge computing (MEC) server. However, if too many IoT devices simultaneously choose to offload the computation tasks to the MEC server via the limited wireless channel, it may lead to the channel congestion, thus increasing time overhead. Facing a large number of IoT devices in smart cities, the centralized resource allocation algorithm needs a lot of signaling exchange, resulting in low efficiency. To solve the problem, this paper studies the joint policy of communication and computing of IoT devices in edge computing through game theory, and proposes distributed Q-learning algorithms with two learning policies. Simulation results show that the algorithm can converge quickly with a balanced solution.

[1]  Feng Xia,et al.  Vehicular Social Networks: Enabling Smart Mobility , 2017, IEEE Communications Magazine.

[2]  Juan Sanchez-Gonzalez,et al.  Power-Efficient Resource Allocation in a Heterogeneous Network With Cellular and D2D Capabilities , 2016, IEEE Transactions on Vehicular Technology.

[3]  Archie C. Chapman,et al.  Convergent Learning Algorithms for Unknown Reward Games , 2013, SIAM J. Control. Optim..

[4]  Yuan Wu,et al.  HybridIoT: Integration of Hierarchical Multiple Access and Computation Offloading for IoT-Based Smart Cities , 2019, IEEE Network.

[5]  Bin Li,et al.  Learning-Based Spectrum Sharing and Spatial Reuse in mm-Wave Ultradense Networks , 2018, IEEE Transactions on Vehicular Technology.

[6]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[7]  Bharadwaj Veeravalli,et al.  Cloud-of-clouds based resource provisioning strategy for continuous write applications , 2017, 2017 23rd Asia-Pacific Conference on Communications (APCC).

[8]  Victor C. M. Leung,et al.  Software-Defined Networks with Mobile Edge Computing and Caching for Smart Cities: A Big Data Deep Reinforcement Learning Approach , 2017, IEEE Communications Magazine.

[9]  Zhigang Chen,et al.  Task Scheduling for Smart City Applications Based on Multi-Server Mobile Edge Computing , 2019, IEEE Access.

[10]  Zhongfeng Wang,et al.  Offloading Optimization for Low-Latency Secure Mobile Edge Computing Systems , 2020, IEEE Wireless Communications Letters.

[11]  Nei Kato,et al.  Optimal Edge Resource Allocation in IoT-Based Smart Cities , 2019, IEEE Network.

[12]  Ciprian Dobre,et al.  Data and Task Offloading in Collaborative Mobile Fog-Based Networks , 2019, IEEE Access.

[13]  Giuseppe La Torre,et al.  Solving Critical Events through Mobile Edge Computing: An Approach for Smart Cities , 2016, 2016 IEEE International Conference on Smart Computing (SMARTCOMP).

[14]  Wenzhong Li,et al.  Efficient Multi-User Computation Offloading for Mobile-Edge Cloud Computing , 2015, IEEE/ACM Transactions on Networking.

[15]  Setareh Maghsudi,et al.  Mobile Edge Computation Offloading Using Game Theory and Reinforcement Learning , 2017, ArXiv.

[16]  Michele Rossi,et al.  On the Allocation of Computing Tasks under QoS Constraints in Hierarchical MEC Architectures , 2019, 2019 Fourth International Conference on Fog and Mobile Edge Computing (FMEC).

[17]  Akshay Shinde,et al.  IoT for smart city: Improvising smart environment , 2017, 2017 International Conference on Energy, Communication, Data Analytics and Soft Computing (ICECDS).

[18]  Krzysztof R. Apt,et al.  A classification of weakly acyclic games , 2012, SAGT.

[19]  Ian Taylor,et al.  SWITCH workbench: A novel approach for the development and deployment of time-critical microservice-based cloud-native applications , 2019, Future Gener. Comput. Syst..

[20]  L. Shapley,et al.  Potential Games , 1994 .

[21]  Tarik Taleb,et al.  Mobile Edge Computing Potential in Making Cities Smarter , 2017, IEEE Communications Magazine.

[22]  Panagiotis Tsarchopoulos,et al.  Smart cities and cloud computing: lessons from the STORM CLOUDS experiment , 2016 .

[23]  Rocio Maciel,et al.  A Use Case in Cybersecurity based in Blockchain to deal with the security and privacy of citizens and Smart Cities Cyberinfrastructures , 2018, 2018 IEEE International Smart Cities Conference (ISC2).

[24]  Antonio Esposito,et al.  Internet of things reference architectures, security and interoperability: A survey , 2018, Internet Things.

[25]  Ciprian Dobre,et al.  Event-based sensor data exchange and fusion in the Internet of Things environments , 2018, J. Parallel Distributed Comput..

[26]  Aawatif Hayar,et al.  Evaluation of the concept of the smart city through local regulation and the importance of local initiative , 2018, 2018 IEEE International Smart Cities Conference (ISC2).

[27]  Hyundong Shin,et al.  Learning for Computation Offloading in Mobile Edge Computing , 2018, IEEE Transactions on Communications.

[28]  L. Shapley,et al.  REGULAR ARTICLEPotential Games , 1996 .

[29]  Huan Zhou,et al.  V2V Data Offloading for Cellular Network Based on the Software Defined Network (SDN) Inside Mobile Edge Computing (MEC) Architecture , 2018, IEEE Access.

[30]  Zhu Han,et al.  Game Theory in Wireless and Communication Networks: Theory, Models, and Applications , 2011 .