An intelligent task offloading algorithm (iTOA) for UAV edge computing network

Abstract Unmanned Aerial Vehicle (UAV) has emerged as a promising technology for the support of human activities, such as target tracking, disaster rescue, and surveillance. However, these tasks require a large computation load of image or video processing, which imposes enormous pressure on the UAV computation platform. To solve this issue, in this work, we propose an intelligent Task Offloading Algorithm (iTOA) for UAV edge computing network. Compared with existing methods, iTOA is able to perceive the network’s environment intelligently to decide the offloading action based on deep Monte Calor Tree Search (MCTS), the core algorithm of Alpha Go. MCTS will simulate the offloading decision trajectories to acquire the best decision by maximizing the reward, such as lowest latency or power consumption. To accelerate the search convergence of MCTS, we also proposed a splitting Deep Neural Network (sDNN) to supply the prior probability for MCTS. The sDNN is trained by a self-supervised learning manager. Here, the training data set is obtained from iTOA itself as its own teacher. Compared with game theory and greedy search-based methods, the proposed iTOA improves service latency performance by 33% and 60%, respectively.

[1]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[2]  Guochu Shou,et al.  Mobile Edge Computing: Progress and Challenges , 2016, 2016 4th IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (MobileCloud).

[3]  Wahidah Hashim,et al.  Channel selection in multi-hop cognitive radio network using reinforcement learning: An experimental study , 2014 .

[4]  Hiroyuki Koga,et al.  A bandwidth allocation scheme to meet flow requirements in mobile edge computing , 2017, 2017 IEEE 6th International Conference on Cloud Networking (CloudNet).

[5]  David Cote,et al.  Using machine learning in communication networks [Invited] , 2018, IEEE/OSA Journal of Optical Communications and Networking.

[6]  Joaquim B. Cavalcante Neto,et al.  Evaluating Competition in Training of Deep Reinforcement Learning Agents in First-Person Shooter Games , 2018, 2018 17th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames).

[7]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[8]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[9]  Mehdi Bennis,et al.  Optimized Computation Offloading Performance in Virtual Edge Computing Systems Via Deep Reinforcement Learning , 2018, IEEE Internet of Things Journal.

[10]  K. B. Letaief,et al.  A Survey on Mobile Edge Computing: The Communication Perspective , 2017, IEEE Communications Surveys & Tutorials.

[11]  Shin-Ming Cheng,et al.  eNB Selection for Machine Type Communications Using Reinforcement Learning Based Markov Decision Process , 2017, IEEE Transactions on Vehicular Technology.

[12]  Soumaya Cherkaoui,et al.  A Game Theory Based Efficient Computation Offloading in an UAV Network , 2019, IEEE Transactions on Vehicular Technology.

[13]  Ole-Christoffer Granmo,et al.  Deep RTS: A Game Environment for Deep Reinforcement Learning in Real-Time Strategy Games , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[14]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Qi Zhang,et al.  Cooperative Computation Offloading for UAVs: A Joint Radio and Computing Resource Allocation Approach , 2018, 2018 IEEE International Conference on Edge Computing (EDGE).

[16]  Liang Xiao,et al.  Learning-Based Privacy-Aware Offloading for Healthcare IoT With Energy Harvesting , 2019, IEEE Internet of Things Journal.

[17]  Wei Cao,et al.  Intelligent Offloading in Multi-Access Edge Computing: A State-of-the-Art Review and Framework , 2019, IEEE Communications Magazine.

[18]  Gang Feng,et al.  iRAF: A Deep Reinforcement Learning Approach for Collaborative Mobile Edge Computing IoT Networks , 2019, IEEE Internet of Things Journal.

[19]  Tapani Ristaniemi,et al.  Learn to Cache: Machine Learning for Network Edge Caching in the Big Data Era , 2018, IEEE Wireless Communications.

[20]  George V. Tsoulos,et al.  Path Loss characteristics for UAV-to-Ground Wireless Channels , 2019, 2019 13th European Conference on Antennas and Propagation (EuCAP).

[21]  Marco Levorato,et al.  Optimal Computation Offloading in Edge-Assisted UAV Systems , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[22]  Jiawei Han,et al.  A Distributed Game Methodology for Crowdsensing in Uncertain Wireless Scenario , 2020, IEEE Transactions on Mobile Computing.

[23]  Shengli Fu,et al.  Intelligent Massive MIMO Antenna Selection Using Monte Carlo Tree Search , 2019, IEEE Transactions on Signal Processing.

[24]  Bin Cao,et al.  Lyapunov Optimization-Based Trade-Off Policy for Mobile Cloud Offloading in Heterogeneous Wireless Networks , 2019, IEEE Transactions on Cloud Computing.

[25]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[26]  Ilyas Alper Karatepe,et al.  Big data caching for networking: moving from cloud to edge , 2016, IEEE Communications Magazine.

[27]  Yueyun Chen,et al.  An Improved Call Admission Control Scheme Based on Reinforcement Learning for Multimedia Wireless Networks , 2009, 2009 International Conference on Wireless Networks and Information Systems.

[28]  Mohsen Guizani,et al.  Unmanned Aerial Vehicles (UAVs): A Survey on Civil Applications and Key Research Challenges , 2018, IEEE Access.

[29]  Shahid Mumtaz,et al.  When Internet of Things Meets Blockchain: Challenges in Distributed Consensus , 2019, IEEE Network.

[30]  Weihua Zhuang,et al.  Learning-Based Computation Offloading for IoT Devices With Energy Harvesting , 2017, IEEE Transactions on Vehicular Technology.

[31]  Rose Qingyang Hu,et al.  Mobility-Aware Edge Caching and Computing in Vehicle Networks: A Deep Reinforcement Learning , 2018, IEEE Transactions on Vehicular Technology.

[32]  Richard D. Gitlin,et al.  Unsupervised machine learning in 5G networks for low latency communications , 2017, 2017 IEEE 36th International Performance Computing and Communications Conference (IPCCC).

[33]  Xuejun Li,et al.  Deep reinforcement learning policy in Hex game system , 2018, 2018 Chinese Control And Decision Conference (CCDC).

[34]  Qi Hao,et al.  Deep Learning for Intelligent Wireless Networks: A Comprehensive Survey , 2018, IEEE Communications Surveys & Tutorials.

[35]  Jin Chen,et al.  Network-connected UAV communications: Potentials and challenges , 2018, China Communications.

[36]  Budhaditya Bhattacharyya,et al.  A Study on the Integration of Machine Learning in Wireless Communication , 2018, 2018 International Conference on Communication and Signal Processing (ICCSP).

[37]  Yun Li,et al.  Joint Optimization of Radio and Virtual Machine Resources With Uncertain User Demands in Mobile Cloud Computing , 2018, IEEE Transactions on Multimedia.

[38]  Wei Liu,et al.  Adaptive congestion avoidance scheme based on reinforcement learning for wireless sensor network , 2011 .

[39]  Li Zhou,et al.  Energy-Latency Tradeoff for Energy-Aware Offloading in Mobile Edge Computing Networks , 2018, IEEE Internet of Things Journal.

[40]  Zibin Zheng,et al.  Joint Computation Offloading and Routing Optimization for UAV-Edge-Cloud Computing Environments , 2018, 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI).