Reinforcement Learning for 5G Caching with Dynamic Cost

In next generation cellular networks (5G) the access points (APs) are anticipated to be equipped with storage devices to serve locally requests for reusable popular contents by caching them at the edge of the network. The ultimate goal is to shift part of the load on the back-haul links from on-peak to off-peak periods, contributing to a better overall network performance and service experience. In order to enable the APs with efficient (optimal) fetch-cache decision making schemes able to work in dynamic settings, we introduce simple but flexible generic time-varying fetching and caching costs, which are then used to formulate a constrained minimization of the aggregate cost across files and time. Since caching decisions in every time slot influence the content availability in future instants, the novel formulation for optimal fetch-cache decisions falls into the class of dynamic programming, for which efficient reinforcement-learning-based solvers are proposed. The performance of our algorithms is assessed via numerical tests, and discussions on the inherent fetching-versus-caching trade-off are provided.

[1]  Georgios B. Giannakis,et al.  DGLB: Distributed Stochastic Geographical Load Balancing over Cloud Networks , 2017, IEEE Transactions on Parallel and Distributed Systems.

[2]  Li Fan,et al.  Web caching and Zipf-like distributions: evidence and implications , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[3]  Leandros Tassiulas,et al.  Resource Allocation and Cross-Layer Control in Wireless Networks , 2006, Found. Trends Netw..

[4]  Deniz Gündüz,et al.  Learning-based optimization of cache content in a small cell base station , 2014, 2014 IEEE International Conference on Communications (ICC).

[5]  Giuseppe Caire,et al.  Wireless caching: technical misconceptions and business barriers , 2016, IEEE Communications Magazine.

[6]  Anja Klein,et al.  Context-Aware Proactive Content Caching With Service Differentiation in Wireless Networks , 2016, IEEE Transactions on Wireless Communications.

[7]  R. Michael Buehrer,et al.  Learning distributed caching strategies in small cell networks , 2014, 2014 11th International Symposium on Wireless Communications Systems (ISWCS).

[8]  Luis M. Lopez-Ramos,et al.  Jointly optimal sensing and resource allocation for multiuser interweave cognitive radios , 2012, 2012 7th International ICST Conference on Cognitive Radio Oriented Wireless Networks and Communications (CROWNCOM).

[9]  Alireza Sadeghi,et al.  Optimal and Scalable Caching for 5G Using Reinforcement Learning of Space-Time Popularities , 2017, IEEE Journal of Selected Topics in Signal Processing.

[10]  Mihaela van der Schaar,et al.  Trend-Aware Video Caching Through Online Learning , 2016, IEEE Transactions on Multimedia.

[11]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.