Caching With Finite Buffer and Request Delay Information: A Markov Decision Process Approach

Edge caching has become a promising technology in future wireless networks owing to its remarkable ability to reduce peak data traffic. However, the storage resource can be limited in practice hence only a small amount of files can be cached. How to improve the cache hit ratio in finite-buffer caching based on the prediction of user demands has become an important problem. In this paper, we study caching policies with finite buffer by exploiting the prediction of a user’s request time, referred to as request delay information (RDI). Based on RDI, we maximize the average cache hit ratio through a Markov decision process (MDP) approach. Specifically, we formulate an MDP problem and apply a modified value iteration algorithm to find an optimal caching policy. Moreover, we provide an upper bound and a lower bound for the cache hit ratio, as well as an analytical cache hit ratio with small buffers. To address the issue that the state space can be prohibitively large in practice, we present a low-complexity heuristic caching policy that is shown to be asymptotically optimal. Simulation results show that introducing RDI may bring significant cache hit ratio gain when the buffer size is limited.

[1]  Jingjing Yao,et al.  On Mobile Edge Caching , 2019, IEEE Communications Surveys & Tutorials.

[2]  Deniz Gündüz,et al.  A Reinforcement-Learning Approach to Proactive Caching in Wireless Networks , 2017, IEEE Journal on Selected Areas in Communications.

[3]  Kai Lai Chung,et al.  A Course in Probability Theory , 1949 .

[4]  Xiaohu You,et al.  User Preference Learning-Based Edge Caching for Fog Radio Access Network , 2018, IEEE Transactions on Communications.

[5]  Huaiyu Dai,et al.  A Survey on Low Latency Towards 5G: RAN, Core Network and Caching Solutions , 2017, IEEE Communications Surveys & Tutorials.

[6]  Wei Chen,et al.  Maximizing Hit Ratio in Finite-Buffer Caching with Request Delay Information: An MDP Approach , 2019, 2019 IEEE Global Communications Conference (GLOBECOM).

[7]  H. Vincent Poor,et al.  Big Data Driven Wireless Communications: A Human-in-the-Loop Pushing Technique for 5G Systems , 2018, IEEE Wireless Communications.

[8]  Alexandros G. Dimakis,et al.  FemtoCaching: Wireless Content Delivery Through Distributed Caching Helpers , 2013, IEEE Transactions on Information Theory.

[9]  Ning Zhang,et al.  Online Proactive Caching in Mobile Edge Computing Using Bidirectional Deep Recurrent Neural Network , 2019, IEEE Internet of Things Journal.

[10]  Urs Niesen,et al.  Fundamental limits of caching , 2012, 2013 IEEE International Symposium on Information Theory.

[11]  Thrasyvoulos Spyropoulos,et al.  Soft Cache Hits: Improving Performance Through Recommendation and Delivery of Related Content , 2018, IEEE Journal on Selected Areas in Communications.

[12]  Jian Li,et al.  Accurate Learning or Fast Mixing? Dynamic Adaptability of Caching Algorithms , 2017, IEEE Journal on Selected Areas in Communications.

[13]  Andreas F. Molisch,et al.  Individual Preference Probability Modeling and Parameterization for Video Content in Wireless Caching Networks , 2019, IEEE/ACM Transactions on Networking.

[14]  Hyundong Shin,et al.  Content-Aware Proactive Caching for Backhaul Offloading in Cellular Network , 2018, IEEE Transactions on Wireless Communications.

[15]  Ying Cui,et al.  Joint Pushing and Caching for Bandwidth Utilization Maximization in Wireless Networks , 2017, GLOBECOM 2017 - 2017 IEEE Global Communications Conference.

[16]  Wei Chen,et al.  The Roadmap to 6G: AI Empowered Wireless Networks , 2019, IEEE Communications Magazine.

[17]  Branka Vucetic,et al.  Localized Small Cell Caching: A Machine Learning Approach Based on Rating Data , 2019, IEEE Transactions on Communications.

[18]  Gang Feng,et al.  Multi-Agent Reinforcement Learning for Efficient Content Caching in Mobile D2D Networks , 2019, IEEE Transactions on Wireless Communications.

[19]  Zhu Han,et al.  Joint Optimization of Caching, Computing, and Radio Resources for Fog-Enabled IoT Using Natural Actor–Critic Deep Reinforcement Learning , 2019, IEEE Internet of Things Journal.

[20]  Tapani Ristaniemi,et al.  Learn to Cache: Machine Learning for Network Edge Caching in the Big Data Era , 2018, IEEE Wireless Communications.

[21]  Symeon Chatzinotas,et al.  A Bayesian Poisson–Gaussian Process Model for Popularity Learning in Edge-Caching Networks , 2019, IEEE Access.

[22]  H. Vincent Poor,et al.  Caching With Time Domain Buffer Sharing , 2019, IEEE Transactions on Communications.

[23]  H. Vincent Poor,et al.  Content Pushing With Request Delay Information , 2017, IEEE Transactions on Communications.

[24]  Vikram Krishnamurthy,et al.  Adaptive Scheme for Caching YouTube Content in a Cellular Network: Machine Learning Approach , 2017, IEEE Access.

[25]  Jun Li,et al.  Distributed Caching for Data Dissemination in the Downlink of Heterogeneous Networks , 2015, IEEE Transactions on Communications.

[26]  Osvaldo Simeone,et al.  Online Edge Caching and Wireless Delivery in Fog-Aided Networks With Dynamic Content Popularity , 2017, IEEE Journal on Selected Areas in Communications.

[27]  Mohamad Assaad,et al.  Energy Efficiency in Cache-Enabled Small Cell Networks With Adaptive User Clustering , 2018, IEEE Transactions on Wireless Communications.

[28]  Alireza Sadeghi,et al.  Optimal and Scalable Caching for 5G Using Reinforcement Learning of Space-Time Popularities , 2017, IEEE Journal of Selected Topics in Signal Processing.

[29]  Kaibin Huang,et al.  Cache-Enabled Heterogeneous Cellular Networks: Optimal Tier-Level Content Placement , 2016, IEEE Transactions on Wireless Communications.

[30]  Zhu Han,et al.  A Stackelberg Game Approach to Proactive Caching in Large-Scale Mobile Edge Networks , 2018, IEEE Transactions on Wireless Communications.

[31]  Georgios B. Giannakis,et al.  Deep Reinforcement Learning for Adaptive Caching in Hierarchical Content Delivery Networks , 2019, IEEE Transactions on Cognitive Communications and Networking.

[32]  Giuseppe Caire,et al.  Wireless caching: technical misconceptions and business barriers , 2016, IEEE Communications Magazine.

[33]  H. Vincent Poor,et al.  Energy Efficient Pushing in AWGN Channels Based on Content Request Delay Information , 2018, IEEE Transactions on Communications.