Caching in Base Station with Recommendation via Q-Learning

Proactive caching in base station (BS) is a promising way to leverage the human-behavior related information to boost the system throughput and improve user experience with low cost. Yet existing caching policies are "blind" to users, i.e., the users are unaware about whether the files to be requested are locally cached at the BS around them, whose performance will degrade in mobile systems due to the randomness of user behavior. For example, when a user stays in a cell where the BS caches the file to be requested later, the user may not initiate the request. When the user sends the request, it may have entered another cell where the BS does not cache the requested file. In this paper, we introduce a simple idea of informing the users about what the BS has cached to make local caching efficient, which can be regarded as a form of recommendation. Since the probabilities that the users request the files are unknown, we resort to Q-learning to perceive the request probability and the statistics of random arrival and departure of mobile users. Then, a cache replacement policy is optimized. Simulation results demonstrate that the proposed policy with recommendation can provide higher long-term system reward than existing policies without recommendation.