DRL based Data Offloading for Intelligent Reflecting Surface Aided Mobile Edge Computing

Recently, the intelligent reflecting surface (IRS) is an emerging and promising technology for achieving higher spectrum and energy efficiency in wireless communication systems. In this paper, we consider a wireless powered mobile edge computing (MEC) network that is equipped with an IRS. The IRS is able to provide a reflecting channel to enhance the offloading capability for edge users. Based on this system model, we investigate an optimisation problem to maximize the sum of users’ utilities, which jointly consider the energy efficiency, time latency, and price of offloading computations. With task offloading, power limited users can complete the computational tasks even when they face data-intensive workloads. However, in a dynamic system, it is complicated to design the optimal offloading decision strategy. To tackle this problem, we propose a deep reinforcement learning (DRL) based approach. In the designed algorithm, in order to get a better reward, the agent chooses a near optimal solution to adjust the workload partitions, the time allocation, and IRS parameters according to the dynamic channel environment and the random arrival of task workload. Numerical results show that the proposed DRL based IRS-aided offloading algorithm can achieve better system performance compared with that without IRS and the relative benchmark algorithms.

[1]  Nan Zhao,et al.  Integrated Networking, Caching, and Computing for Connected Vehicles: A Deep Reinforcement Learning Approach , 2018, IEEE Transactions on Vehicular Technology.

[2]  Jun Zhao,et al.  Deep Reinforcement Learning-Based Intelligent Reflecting Surface for Secure Wireless Communications , 2020, IEEE Transactions on Wireless Communications.

[3]  Arumugam Nallanathan,et al.  Latency Minimization for Intelligent Reflecting Surface Aided Mobile Edge Computing , 2020, IEEE Journal on Selected Areas in Communications.

[4]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[5]  Ying-Chang Liang,et al.  Intelligent Reflecting Surface Assisted Non-Orthogonal Multiple Access , 2019, 2020 IEEE Wireless Communications and Networking Conference (WCNC).

[6]  Yuanyuan Yang,et al.  Energy-efficient dynamic offloading and resource scheduling in mobile cloud computing , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[7]  Derrick Wing Kwan Ng,et al.  Practical Non-Linear Energy Harvesting Model and Resource Allocation for SWIPT Systems , 2015, IEEE Communications Letters.

[8]  Jun Zhao,et al.  Intelligent Reflecting Surface Meets Mobile Edge Computing: Enhancing Wireless Communications for Computation Offloading , 2020 .

[9]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[10]  Ying Jun Zhang,et al.  Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks , 2018, IEEE Transactions on Mobile Computing.

[11]  K. B. Letaief,et al.  A Survey on Mobile Edge Computing: The Communication Perspective , 2017, IEEE Communications Surveys & Tutorials.

[12]  Yi Wang,et al.  Backscatter-Assisted Computation Offloading for Energy Harvesting IoT Devices via Policy-based Deep Reinforcement Learning , 2019, 2019 IEEE/CIC International Conference on Communications Workshops in China (ICCC Workshops).

[13]  Shane Legg,et al.  Noisy Networks for Exploration , 2017, ICLR.

[14]  Changsheng You,et al.  Channel Estimation and Passive Beamforming for Intelligent Reflecting Surface: Discrete Phase Shift and Progressive Refinement , 2020, IEEE Journal on Selected Areas in Communications.

[15]  Arumugam Nallanathan,et al.  Resource Allocation for Intelligent Reflecting Surface Aided Wireless Powered Mobile Edge Computing in OFDM Systems , 2020, IEEE Transactions on Wireless Communications.

[16]  Ying-Chang Liang,et al.  Applications of Deep Reinforcement Learning in Communications and Networking: A Survey , 2018, IEEE Communications Surveys & Tutorials.

[17]  James Gross,et al.  Relaying with finite blocklength: Challenge vs. opportunity , 2016, 2016 IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM).

[18]  Yi Wang,et al.  Backscatter-Aided Hybrid Data Offloading for Mobile Edge Computing via Deep Reinforcement Learning , 2019, ICML 2019.

[19]  Xiao Lu,et al.  Toward Smart Wireless Communications via Intelligent Reflecting Surfaces: A Contemporary Survey , 2019, IEEE Communications Surveys & Tutorials.