Distributed cache replacement for caching-enable base stations in cellular networks

Distributive service data storage at the caching-enabled base stations (BSs) can reduce the traffic load in future cellular networks. Taking the limited caching space into account, it is necessary for the BSs to adjust their caching data based on service popularity in order to achieve better caching efficiency. In this paper, we investigate the cache replacement strategy for BSs to minimize the transmission cost between BSs in cellular networks. The cache replacement problem is modelled as a Markov Decision Process (MDP). Without extra information exchange about caching data between the BSs, we propose a distributed cache replacement strategy based on Q-learning. Especially, we calculate the transmission cost for possible cache replacement actions according to the previous data request and transmission between BSs. The convergence of the proposed distributed cache replacement strategy is proved by sequential stage game model. Simulation results verify the convergence of the proposed cache replacement strategy and show its performance gain compared to conventional strategies.

[1]  Martin F. Arlitt,et al.  Evaluating content management techniques for Web proxy caches , 2000, PERV.

[2]  Tamer A. ElBatt,et al.  Distributed Cooperative Q-Learning for Power Allocation in Cognitive Femtocell Networks , 2012, 2012 IEEE Vehicular Technology Conference (VTC Fall).

[3]  Hans Kleine Büning,et al.  A Convergent Multiagent Reinforcement Learning Approach for a Subclass of Cooperative Stochastic Games , 2011, ALA.

[4]  Sang Lyul Min,et al.  Efficient Replacement of Nonuniform Objects in Web Caches , 2002, Computer.

[5]  Mohamed Hefeeda,et al.  Traffic Modeling and Proportional Partial Caching for Peer-to-Peer Systems , 2008, IEEE/ACM Transactions on Networking.

[6]  Alexandros G. Dimakis,et al.  Network Coding for Distributed Storage Systems , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[7]  Martin F. Arlitt,et al.  Workload characterization of a Web proxy in a cable modem environment , 1999, PERV.

[8]  Wei Wang,et al.  Proactive storage at caching-enable base stations in cellular networks , 2013, 2013 IEEE 24th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC).

[9]  Edward A. Fox,et al.  Caching Proxies: Limitations and Potentials , 1995, WWW.

[10]  Konstantinos Psounis,et al.  A randomized Web-cache replacement scheme , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[11]  Yishay Mansour,et al.  Convergence of Optimistic and Incremental Q-Learning , 2001, NIPS.