DEEP Reinforcement Learning Based Energy Beamforming for Powering Sensor Networks

We focus on a wireless sensor network powered with an energy beacon, where sensors send their measurements to the sink using the harvested energy. The aim of the system is to estimate an unknown signal over the area of interest as accurately as possible. We investigate optimal energy beamforming at the energy beacon and optimal transmit power allocation at the sensors under non-linear energy harvesting models. We use a deep reinforcement learning (RL) based approach where multi-layer neural networks are utilized. We illustrate how RL can approach the optimum performance without explicitly forming a system model, but suffers from slow convergence. We also quantify the importance of the number of antennas at the energy beamformer and the number of sensors.

[1]  Koji Ishibashi,et al.  Wireless Power Transfer for Distributed Estimation in Sensor Networks , 2017, IEEE Journal of Selected Topics in Signal Processing.

[2]  Mani B. Srivastava,et al.  Optimization vs. Reinforcement Learning for Wirelessly Powered Sensor Networks , 2018, 2018 IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[3]  S. R. Searle,et al.  On Deriving the Inverse of a Sum of Matrices , 1981 .

[4]  F. Mezzadri How to generate random matrices from the classical compact groups , 2006, math-ph/0609050.

[5]  Derrick Wing Kwan Ng,et al.  Practical Non-Linear Energy Harvesting Model and Resource Allocation for SWIPT Systems , 2015, IEEE Communications Letters.

[6]  Inkyu Lee,et al.  Joint Transceiver Designs for MSE Minimization in MIMO Wireless Powered Sensor Networks , 2018, IEEE Transactions on Wireless Communications.

[7]  Erik G. Larsson,et al.  Simultaneous Information and Power Transfer for Broadband Wireless Systems , 2012, IEEE Transactions on Signal Processing.

[8]  Bruno Clerckx,et al.  Multiuser Wirelessly Powered Backscatter Communications: Nonlinearity, Waveform Design, and SINR-Energy Tradeoff , 2018, IEEE Transactions on Wireless Communications.

[9]  C. Tellambura,et al.  Linear estimation of correlated data in wireless sensor networks with optimum power allocation and analog modulation , 2008 .

[10]  Bruno Clerckx,et al.  Wireless Information and Power Transfer: Nonlinearity, Waveform Design, and Rate-Energy Tradeoff , 2016, IEEE Transactions on Signal Processing.

[11]  Kaibin Huang,et al.  Energy Harvesting Wireless Communications: A Review of Recent Advances , 2015, IEEE Journal on Selected Areas in Communications.

[12]  Bruno Clerckx,et al.  Waveform Design for Wireless Power Transfer With Limited Feedback , 2017, IEEE Transactions on Wireless Communications.

[13]  Jing Yang,et al.  Transmission with Energy Harvesting Nodes in Fading Wireless Channels: Optimal Policies , 2011, IEEE Journal on Selected Areas in Communications.

[14]  Yao-Win Peter Hong,et al.  Wireless Power Transfer for Distributed Estimation in Wireless Passive Sensor Networks , 2016, IEEE Transactions on Signal Processing.

[15]  Aggelos Bletsas,et al.  Sensitive and Nonlinear Far-Field RF Energy Harvesting in Wireless Communications , 2017, IEEE Transactions on Wireless Communications.

[16]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[17]  Jari Turunen,et al.  Imaging of Gaussian Schell-model sources , 1988 .

[18]  Yi Huang,et al.  A High-Efficiency Broadband Rectenna for Ambient Wireless Energy Harvesting , 2015, IEEE Transactions on Antennas and Propagation.

[19]  Rong Du,et al.  Towards Immortal Wireless Sensor Networks by Optimal Energy Beamforming and Data Routing , 2018, IEEE Transactions on Wireless Communications.

[20]  Peter Henderson,et al.  An Introduction to Deep Reinforcement Learning , 2018, Found. Trends Mach. Learn..

[21]  Mats Viberg,et al.  Simultaneous information and power transfer under a non-linear RF energy harvesting model , 2017, 2017 IEEE International Conference on Communications Workshops (ICC Workshops).

[22]  Mani Srivastava,et al.  Energy-aware wireless microsensor networks , 2002, IEEE Signal Process. Mag..

[23]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[24]  Serdar Yüksel,et al.  Unitary Precoding and Basis Dependency of MMSE Performance for Gaussian Erasure Channels , 2011, IEEE Transactions on Information Theory.