On optimal online power policies for energy harvesting with finite-state Markov channels

We investigate the problem of continuous-time energy harvesting in communication systems operating over fading wireless channels. We model the fading as a finite-state continuous-time Markov process, and the battery dynamics as a storage dam process with reflecting boundary conditions. We describe a set of necessary conditions for the ergodicity of the dam process. Followed by these conditions, we establish an upper bound on the ergodic channel throughput. We further determine some structure for good transmission power policies based on a throughput maximization problem. Specifically, using calculus of variations techniques, we derive Euler-Lagrange equations as a necessary condition for optimal power policies. In the case of a Markov channel with two channel states (i.e. Gilbert-Elliot channel), we characterize power policies by solving these equations numerically.