RLC: A Reinforcement Learning-Based Charging Algorithm for Mobile Devices

Wireless charging has been demonstrated as a promising technology for prolonging device operational lifetimes in Wireless Rechargeable Networks (WRNs). To schedule a mobile charger to move along a predesigned trajectory to charge devices, most existing studies assume that the precise location information of devices is already known. Unfortunately, this assumption does not always hold in real mobile application, because the activities of the vast majority of mobile devices carried by mobile agents appear dynamic and random. To the best of our knowledge, this is the first work to study how to wirelessly charge mobile devices with non-deterministic mobility. We aim to provide effective charging service to them, subject to the energy capacity of the mobile charger. We formalize the effective charging problem as a charging reward maximization problem (CRMP), where the amount of reward obtained by charging a device is inversely proportional to the residual lifetime of the device. Then, we prove that CRMP is NP-hard. To derive an effective charging heuristic, an algorithm based on Reinforcement Learning (RL) is proposed. The evaluation results show that the RL-based charging algorithm achieves excellent charging effectiveness. We further interpret the learned heuristic to gain deep and valuable insights into the design options.

[1]  Jianping Pan,et al.  Evaluating the On-Demand Mobile Charging in Wireless Sensor Networks , 2015, IEEE Transactions on Mobile Computing.

[2]  Xing Xie,et al.  GeoLife: Managing and Understanding Your Past Life over Maps , 2008, The Ninth International Conference on Mobile Data Management (mdm 2008).

[3]  Shaojie Tang,et al.  CHASE: Charging and Scheduling Scheme for Stochastic Event Capture in Wireless Rechargeable Sensor Networks , 2020, IEEE Transactions on Mobile Computing.

[4]  Chaoming Song,et al.  Modelling the scaling properties of human mobility , 2010, 1010.0436.

[5]  Le Song,et al.  2 Common Formulation for Greedy Algorithms on Graphs , 2018 .

[6]  Jie Wu,et al.  Bundle Charging: Wireless Charging Energy Minimization in Dense Wireless Sensor Networks , 2019, 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS).

[7]  Chi Lin,et al.  mTS: Temporal-and Spatial-Collaborative Charging for Wireless Rechargeable Sensor Networks with Multiple Vehicles , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[8]  Weifa Liang,et al.  Maintaining Large-Scale Rechargeable Sensor Networks Perpetually via Multiple Mobile Charging Vehicles , 2016, ACM Trans. Sens. Networks.

[9]  Shan Lin,et al.  Charge me if you can: charging path optimization and scheduling in mobile networks , 2016, MobiHoc.

[10]  Xing Xie,et al.  Mining interesting locations and travel sequences from GPS trajectories , 2009, WWW '09.

[11]  Panlong Yang,et al.  Collaborated Tasks-driven Mobile Charging and Scheduling: A Near Optimal Result , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[12]  Jie Wu,et al.  Homing spread: Community home-based multi-copy routing in mobile social networks , 2013, 2013 Proceedings IEEE INFOCOM.

[13]  Stefano Secci,et al.  Estimating human trajectories and hotspots through mobile phone data , 2014, Comput. Networks.

[14]  Weifa Liang,et al.  Approximation Algorithms for the Team Orienteering Problem , 2020, IEEE INFOCOM 2020 - IEEE Conference on Computer Communications.

[15]  M. Soljačić,et al.  Wireless Power Transfer via Strongly Coupled Magnetic Resonances , 2007, Science.

[16]  Hongyi Wu,et al.  Low-Cost Collaborative Mobile Charging for Large-Scale Wireless Sensor Networks , 2017, IEEE Transactions on Mobile Computing.

[17]  Lei Shi,et al.  Reinforcement Learning for a Novel Mobile Charging Strategy in Wireless Rechargeable Sensor Networks , 2018, WASA.

[18]  Albert-László Barabási,et al.  Limits of Predictability in Human Mobility , 2010, Science.

[19]  Mohammad S. Obaidat,et al.  TSCA: A Temporal-Spatial Real-Time Charging Scheduling Algorithm for On-Demand Architecture in Wireless Rechargeable Sensor Networks , 2018, IEEE Transactions on Mobile Computing.

[20]  Weifa Liang,et al.  Minimizing the Maximum Charging Delay of Multiple Mobile Chargers Under the Multi-Node Energy Charging Scheme , 2021, IEEE Transactions on Mobile Computing.

[21]  Jian Peng,et al.  An Effective Multi-node Charging Scheme for Wireless Rechargeable Sensor Networks , 2020, IEEE INFOCOM 2020 - IEEE Conference on Computer Communications.

[22]  Guihai Chen,et al.  SCAPE: Safe Charging with Adjustable Power , 2014, 2014 IEEE 34th International Conference on Distributed Computing Systems.

[23]  Chi Lin,et al.  3DCS: A 3-D Dynamic Collaborative Scheduling Scheme for Wireless Rechargeable Sensor Networks with Heterogeneous Chargers , 2018, 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS).

[24]  Byoungwoo Kang,et al.  Battery materials for ultrafast charging and discharging , 2009, Nature.

[25]  M. Held,et al.  A dynamic programming approach to sequencing problems , 1962, ACM National Meeting.

[26]  Jianping Pan,et al.  Mobile-to-mobile energy replenishment in mission-critical robotic sensor networks , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[27]  Tang Liu,et al.  A deep reinforcement learning-based on-demand charging algorithm for wireless rechargeable sensor networks , 2021, Ad Hoc Networks.

[28]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[29]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[30]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[31]  Luca Benini,et al.  A survey of design techniques for system-level dynamic power management , 2000, IEEE Trans. Very Large Scale Integr. Syst..

[32]  Thomas H. Clausen,et al.  A Study of LoRa: Long Range & Low Power Networks for the Internet of Things , 2016, Sensors.

[33]  Hanif D. Sherali,et al.  On traveling path and related problems for a mobile station in a rechargeable sensor network , 2013, MobiHoc.

[34]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[35]  Cong Wang,et al.  Wireless Rechargeable Sensor Networks , 2015, SpringerBriefs in Electrical and Computer Engineering.

[36]  M. Held,et al.  A dynamic programming approach to sequencing problems , 1962, ACM National Meeting.

[37]  Jie Wu,et al.  Collaborative Mobile Charging , 2015, IEEE Transactions on Computers.

[38]  Dimitri P. Bertsekas,et al.  Network optimization : continuous and discrete models , 1998 .

[39]  Partha Pratim Pande,et al.  Trading-Off Accuracy and Energy of Deep Inference on Embedded Systems: A Co-Design Approach , 2018, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[40]  Hongyi Wu,et al.  Learning an Effective Charging Scheme for Mobile Devices , 2020, 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[41]  Huadong Ma,et al.  Opportunities in mobile crowd sensing , 2014, IEEE Communications Magazine.

[42]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .