Deep Reinforcement Learning for Mobile Edge Caching: Review, New Features, and Open Issues

Mobile edge caching is a promising technique to reduce network traffic and improve the quality of experience of mobile users. However, mobile edge caching is a challenging decision making problem with unknown future content popularity and complex network characteristics. In this article, we advocate the use of DRL to solve mobile edge caching problems by presenting an overview of recent works on mobile edge caching and DRL. We first examine the key issues in mobile edge caching and review the existing learning- based solutions proposed in the literature. We also discuss the unique features in the application of DRL in mobile edge caching, and illustrate an example of DRL-based mobile edge caching with trace-data-driven simulation results. This article concludes with a discussion of several open issues that call for substantial future research efforts.

[1]  Marc Peter Deisenroth,et al.  Deep Reinforcement Learning: A Brief Survey , 2017, IEEE Signal Processing Magazine.

[2]  Min Sheng,et al.  Learning-Based Content Caching and Sharing for Wireless Networks , 2017, IEEE Transactions on Communications.

[3]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[4]  Ming Wang,et al.  Content-Aware Concurrent Multipath Transfer for High-Definition Video Streaming over Heterogeneous Wireless Networks , 2016, IEEE Transactions on Parallel and Distributed Systems.

[5]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[6]  Vikram Krishnamurthy,et al.  Adaptive Scheme for Caching YouTube Content in a Cellular Network: Machine Learning Approach , 2017, IEEE Access.

[7]  Deniz Gündüz,et al.  Learning-based optimization of cache content in a small cell base station , 2014, 2014 IEEE International Conference on Communications (ICC).

[8]  Mihaela van der Schaar,et al.  Trend-Aware Video Caching Through Online Learning , 2016, IEEE Transactions on Multimedia.

[9]  M. Draief,et al.  Placing dynamic content in caches with small population , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[10]  Tom Schaul,et al.  Dueling Network Architectures for Deep Reinforcement Learning , 2015, ICML.

[11]  Yan Zhang,et al.  Information Caching Strategy for Cyber Social Computing Based Wireless Networks , 2017, IEEE Transactions on Emerging Topics in Computing.

[12]  Wei Wang,et al.  Edge Caching at Base Stations With Device-to-Device Offloading , 2017, IEEE Access.

[13]  Mehdi Bennis,et al.  Living on the edge: The role of proactive caching in 5G wireless networks , 2014, IEEE Communications Magazine.

[14]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[15]  Yan Zhang,et al.  Cooperative Content Caching in 5G Networks with Mobile Edge Computing , 2018, IEEE Wireless Communications.