Multiple Timescale Energy Scheduling for Wireless Communication with Energy Harvesting Devices

The primary challenge in wireless communica- tion with energy harvesting devices is to efficiently utilize the harvesting energy such that the data packet transmis- sion could be supported. This challenge stems from not only QoS requirement imposed by the wireless communica- tion application, but also the energy harvesting dynamics and the limited battery capacity. Traditional solar predict- able energy harvesting models are perturbed by prediction errors, which could deteriorate the energy management algorithms based on this model. To cope with these issues, we first propose in this paper a non-homogenous Markov chain model based on experimental data, which can accu- rately describe the solar energy harvesting process in contrast to traditional predictable energy models. Due to different timescale between the energy harvesting process and the wireless data transmission process, we propose a general framework of multiple timescale Markov deci- sion process (MMDP) model to formulate the joint energy scheduling and transmission control problem under differ- ent timescales. We then derive the optimal control policies via a joint dynamic programming and value iteration approach. Extensive simulations are carried out to study the performances of the proposed schemes.