Deep Learning Based Online Power Control for Large Energy Harvesting Networks

In this paper, we propose a deep learning based approach to design online power control policies for large EH networks, which are often intractable stochastic control problems. In the proposed approach, for a given EH network, the optimal on-line power control rule is learned by training a deep neural network (DNN), using the solution of offline policy design problem. Under the proposed scheme, in a given time slot, the transmit power is obtained by feeding the current system state to the trained DNN. Our results illustrate that the DNN based online power control scheme outperforms a Markov decision process based policy. In general, the proposed deep learning based approach can be used to find solutions to large intractable stochastic control problems.

[1]  Sennur Ulukus,et al.  Optimal Policies for Wireless Networks With Energy Harvesting Transmitters and Receivers: Effects of Decoding Costs , 2015, IEEE Journal on Selected Areas in Communications.

[2]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[3]  Jakob Hoydis,et al.  An Introduction to Deep Learning for the Physical Layer , 2017, IEEE Transactions on Cognitive Communications and Networking.

[4]  Chandra R. Murthy,et al.  Distributed Power Control for Multi-Hop Energy Harvesting Links With Retransmission , 2018, IEEE Transactions on Wireless Communications.

[5]  Sennur Ulukus,et al.  Energy Harvesting Multiple Access Channels: Optimal and Near-Optimal Online Policies , 2018, IEEE Transactions on Communications.

[6]  Andrea Zanella,et al.  Long-range communications in unlicensed bands: the rising stars in the IoT and smart city scenarios , 2015, IEEE Wireless Communications.

[7]  Umberto Spagnolini,et al.  Optimality of myopic scheduling and whittle indexability for energy harvesting sensors , 2012, 2012 46th Annual Conference on Information Sciences and Systems (CISS).

[8]  E Weinan,et al.  Deep Learning Approximation for Stochastic Control Problems , 2016, ArXiv.

[9]  Sakshi Kapoor,et al.  Distributed Scheduling Schemes in Energy Harvesting Multiple Access , 2017, IEEE Wireless Communications Letters.

[10]  Aylin Yener,et al.  Optimum Transmission Policies for Battery Limited Energy Harvesting Nodes , 2010, IEEE Transactions on Wireless Communications.

[11]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[12]  Jingxian Wu,et al.  Online throughput maximization in an energy harvesting multiple access channel with fading , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[13]  Deniz Gündüz,et al.  Multi-Access Communications With Energy Harvesting: A Multi-Armed Bandit Model and the Optimality of the Myopic Policy , 2015, IEEE Journal on Selected Areas in Communications.

[14]  K. J. Ray Liu,et al.  Advances in Energy Harvesting Communications: Past, Present, and Future Challenges , 2016, IEEE Communications Surveys & Tutorials.

[15]  Zhi Chen,et al.  Intelligent Power Control for Spectrum Sharing: A Deep Reinforcement Learning Approach , 2017, ArXiv.

[16]  Xiaodong Wang,et al.  Iterative Dynamic Water-Filling for Fading Multiple-Access Channels With Energy Harvesting , 2014, IEEE Journal on Selected Areas in Communications.

[17]  Wei Chen,et al.  On Throughput Maximization of Time Division Multiple Access With Energy Harvesting Users , 2016, IEEE Transactions on Vehicular Technology.