Persistent surveillance of events with unknown, time-varying statistics

We consider the problem of monitoring stochastic, time-varying events occurring at discrete locations. Our problem formulation extends prior work in persistent surveillance by considering the objective of maximizing event detections in unknown, dynamic environments where the rates of events are time-inhomogeneous and may be subject to abrupt changes. We propose a novel monitoring algorithm that effectively strikes a balance between exploration and exploitation as well as a balance between remembering and discarding information to handle temporal variations in unknown environments. We present an analysis proving the long-run average optimality of the policies generated by our algorithm under the assumption that the total temporal variations are sub-linear. We present simulation results demonstrating the effectiveness of our algorithm in several monitoring scenarios inspired by real-world applications, and its robustness to both continuous-random and abrupt changes in the statistics of the observed processes.

[1]  Mac Schwager,et al.  Generating informative paths for persistent sensing in unknown environments , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Ying He,et al.  Sensor scheduling for target tracking in sensor networks , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[3]  Nathan Michael,et al.  Persistent surveillance with a team of MAVs , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Alfred O. Hero,et al.  Information Theoretic Approaches to Sensor Management , 2008 .

[5]  Mac Schwager,et al.  Correlated Orienteering Problem and its application to informative path planning for persistent monitoring tasks , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  Gilbert Laporte,et al.  The orienteering problem with variable profits , 2013, Networks.

[7]  Mac Schwager,et al.  Persistent Robotic Tasks: Monitoring and Sweeping in Changing Environments , 2011, IEEE Transactions on Robotics.

[8]  Keith D. Kastella,et al.  Foundations and Applications of Sensor Management , 2010 .

[9]  Geoffrey A. Hollinger,et al.  Underwater Data Collection Using Robotic Sensor Networks , 2012, IEEE Journal on Selected Areas in Communications.

[10]  Daniela Rus,et al.  Persistent monitoring of events with stochastic arrivals at multiple stations , 2013, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Yuval Peres,et al.  Bandits with switching costs: T2/3 regret , 2013, STOC.

[12]  Munther A. Dahleh,et al.  Continuous path planning for a data harvesting mobile server , 2008, 2008 47th IEEE Conference on Decision and Control.

[13]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[14]  Christos G. Cassandras,et al.  An Optimal Control Approach to the Multi-Agent Persistent Monitoring Problem , 2012, IEEE Transactions on Automatic Control.

[15]  Mac Schwager,et al.  Decentralized, Adaptive Coverage Control for Networked Robots , 2009, Int. J. Robotics Res..

[16]  Daniela Rus,et al.  Anytime planning of optimal schedules for a mobile sensing robot , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Francesco Bullo,et al.  Stochastic surveillance strategies for spatial quickest detection , 2011, IEEE Conference on Decision and Control and European Control Conference.

[18]  Omar Besbes,et al.  Non-Stationary Stochastic Optimization , 2013, Oper. Res..

[19]  Geoffrey A. Hollinger,et al.  Sampling-based Motion Planning for Robotic Information Gathering , 2013, Robotics: Science and Systems.

[20]  Gaurav S. Sukhatme,et al.  Informative path planning for an autonomous underwater vehicle , 2010, 2010 IEEE International Conference on Robotics and Automation.

[21]  Eric Moulines,et al.  On Upper-Confidence Bound Policies for Switching Bandit Problems , 2011, ALT.

[22]  Gaurav S. Sukhatme,et al.  Persistent ocean monitoring with underwater gliders: Adapting sampling resolution , 2011, J. Field Robotics.

[23]  Hoong Chuin Lau,et al.  Orienteering Problem: A survey of recent variants, solution approaches and applications , 2016, Eur. J. Oper. Res..

[24]  Vaibhav Srivastava,et al.  Surveillance in an abruptly changing world via multiarmed bandits , 2014, 53rd IEEE Conference on Decision and Control.

[25]  Sébastien Bubeck,et al.  Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems , 2012, Found. Trends Mach. Learn..

[26]  Daniela Rus,et al.  Persistent Surveillance of Events with Unknown Rate Statistics , 2016, WAFR.

[27]  Mac Schwager,et al.  Planning periodic persistent monitoring trajectories for sensing robots in Gaussian Random Fields , 2013, 2013 IEEE International Conference on Robotics and Automation.

[28]  William K. Durfee,et al.  IEEE/RSJ/GI International Conference on Intelligent Robots and Systems , 1994 .

[29]  Peter Auer,et al.  UCB revisited: Improved regret bounds for the stochastic multi-armed bandit problem , 2010, Period. Math. Hung..

[30]  Elon Rimon,et al.  Competitive on-line coverage of grid environments by a mobile robot , 2003, Comput. Geom..

[31]  Elaheh Fata,et al.  Persistent monitoring in discrete environments: Minimizing the maximum weighted latency between observations , 2012, Int. J. Robotics Res..