Dynamic Bandwidth Allocation Scheme for Wireless Networks with Energy Harvesting Using Actor-Critic Deep Reinforcement Learning

In this paper, we propose an efficient bandwidth allocation scheme in heterogeneous wireless networks with a single macro-cell base station (MBS) and several small-cell base stations (SBSs) that are powered by solar energy harvesters. This paper aims to design an actor-critic deep reinforcement learning (RL) agent at the MBS (i.e. the main controller) with the purpose of maximizing user satisfaction ratio and energy efficiency in the network. The RL agent learns the stochastic arrivals of traffic requests and harvested energy through direct interaction with the network environment and thus can obtain the optimal bandwidth allocation policy in order to enhance network sustainability and performance. For this purpose, we first formulate the bandwidth allocation problem as the framework of a Markov decision process, and then, employ the actor-critic RL algorithm to find the optimal policy for bandwidth allocation. The actor and the critic of the RL agent use deep neural network to approximate the policy function and the value function, respectively. More specifically, the actor generates action based on the output of the policy network while the critic helps the actor evaluate the policy by using the value network. Simulation results are shown to illustrate the performance of the proposed scheme.