Geometric Reinforcement Learning Based Path Planning for Mobile Sensor Networks in Advection-Diffusion Field Reconstruction

We propose a geometric reinforcement learning algorithm for real-time path planning for mobile sensor networks (MSNs) in the problem of reconstructing a spatial-temporal varying field described by the advection-diffusion partial differential equation. A Luenberger state estimator is provided to reconstruct the concentration field, which uses the collected measurements from a MSN along its trajectory. Since the path of the MSN is critical in reconstructing the field, a novel geometric reinforcement learning (GRL) algorithm is developed for the real-time path planning. The basic idea of GRL is to divide the whole area into a series of lattice to employ a specific time-varying reward matrix, which contains the information of the length of path and the mapping error. Thus, the proposed GRL can balance the performance of the field reconstruction and the efficiency of the path. By updating the reward matrix, the real-time path planning problem can be converted to the shortest path problem in a weighted graph, which can be solved efficiently using dynamic programming. The convergence of calculating the reward matrix is theoretically proven. Simulation results serve to demonstrate the effectiveness and feasibility of the proposed GRL for a MSN.

[1]  Lingzhong Guo,et al.  Estimation of spatial derivatives and identification of continuous spatio-temporal dynamical systems , 2006 .

[2]  Fumin Zhang,et al.  Robust Cooperative Exploration With a Switching Strategy , 2012, IEEE Transactions on Robotics.

[3]  Michael A. Demetriou,et al.  Coupled Controls-Computational Fluids Approach for the Estimation of the Concentration From a Moving Gaseous Source in a 2-D Domain With a Lyapunov-Guided Sensing Aerial Vehicle , 2014, IEEE Transactions on Control Systems Technology.

[4]  Ümit Özgüner,et al.  Motion planning for multitarget surveillance with mobile sensor agents , 2005, IEEE Transactions on Robotics.

[5]  Francesco Bullo,et al.  Optimal sensor placement and motion coordination for target tracking , 2006, Autom..

[6]  Fumin Zhang,et al.  Cooperative parameter identification of advection-diffusion processes using a mobile sensor network , 2017, 2017 American Control Conference (ACC).

[7]  David M. Fratantoni,et al.  Multi-AUV Control and Adaptive Sampling in Monterey Bay , 2006, IEEE Journal of Oceanic Engineering.

[8]  Maciej Patan,et al.  Sensor network design for the estimation of spatially distributed processes , 2010, Int. J. Appl. Math. Comput. Sci..

[9]  Wanquan Liu,et al.  Geometric Reinforcement Learning for Path Planning of UAVs , 2015, J. Intell. Robotic Syst..

[10]  Wencen Wu,et al.  Online Passive Identifier for Spatially Distributed Systems Using Mobile Sensor Networks , 2017, IEEE Transactions on Control Systems Technology.

[11]  Wencen Wu,et al.  Sensing-motion co-planning for reconstructing a spatially distributed field using a mobile sensor network , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[12]  Naomi Ehrich Leonard,et al.  Cooperative Filters and Control for Cooperative Exploration , 2010, IEEE Transactions on Automatic Control.

[13]  Siddhartha S. Srinivasa,et al.  Decentralized estimation and control of graph connectivity in mobile sensor networks , 2008, ACC.

[14]  Michael A. Demetriou,et al.  Guidance of Mobile Actuator-Plus-Sensor Networks for Improved Control and Estimation of Distributed Parameter Systems , 2010, IEEE Transactions on Automatic Control.

[15]  Luigi Fortuna,et al.  Reinforcement Learning and Adaptive Dynamic Programming for Feedback Control , 2009 .

[16]  Fumin Zhang,et al.  Cooperative filtering for parameter identification of diffusion processes , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[17]  R. Olfati-Saber,et al.  Consensus Filters for Sensor Networks and Distributed Sensor Fusion , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[18]  D. Ucinski Optimal measurement methods for distributed parameter system identification , 2004 .