Event-based sensor activation for indoor occupant distribution estimation

The information of the distribution of occupant in an indoor environment is important for building energy saving under normal conditions and for evacuation under emergent conditions, and thus is of great practical interest. Due to low set-up cost, wireless sensor networks powered by batteries are usually used for such estimation. The question is how to activate the sensors to minimize the estimation error within a given period of time. In this paper we develop an event-based activation policy which can be easily implemented in a decentralized way. We use numerical experiments to compare this policy with four other policies and to demonstrate the impact of various factors on the performance of these policies, such as the topology, sensor accuracy, battery capacity, occupant movement model, and occupant population. Our method outperforms the other policies in all the tested scenarios.

[1]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[2]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[3]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[4]  John N. Tsitsiklis,et al.  Rollout Algorithms for Combinatorial Optimization , 1997, J. Heuristics.

[5]  D.A. Castanon,et al.  Rollout Algorithms for Stochastic Scheduling Problems , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[6]  Edwin R. Galea,et al.  A review of the methodologies used in the computer simulation of evacuation from the built environment , 1999 .

[7]  Christos G. Cassandras,et al.  Introduction to Discrete Event Systems , 1999, The Kluwer International Series on Discrete Event Dynamic Systems.

[8]  David Beymer,et al.  Person counting using stereo , 2000, Proceedings Workshop on Human Motion.

[9]  B. Krogh,et al.  State aggregation in Markov decision processes , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[10]  Zhiyuan Ren,et al.  A time aggregation approach to Markov decision processes , 2002, Autom..

[11]  S. Marcus,et al.  Approximate receding horizon approach for Markov decision processes: average reward case , 2003 .

[12]  Xi-Ren Cao,et al.  A basic formula for online policy gradient algorithms , 2005, IEEE Transactions on Automatic Control.

[13]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[14]  Anshul Kumar,et al.  Battery model for embedded systems , 2005, 18th International Conference on VLSI Design held jointly with 4th International Conference on Embedded Systems Design.

[15]  Xi-Ren Cao,et al.  Stochastic learning and optimization - A sensitivity-based approach , 2007, Annu. Rev. Control..

[16]  Cristiano Cervellera,et al.  Neural network and regression spline value function approximations for stochastic dynamic programming , 2007, Comput. Oper. Res..

[17]  Xi Chen,et al.  Design of a randomly distributed sensor network for target detection , 2007, Autom..

[18]  Peter B. Luh,et al.  Incremental Value Iteration for Time-Aggregated Markov-Decision Processes , 2007, IEEE Transactions on Automatic Control.

[19]  Warren B. Powell,et al.  Approximate Dynamic Programming - Solving the Curses of Dimensionality , 2007 .

[20]  Jie Song,et al.  Infrared passenger flow collection system based on RBF neural net , 2008, 2008 International Conference on Machine Learning and Cybernetics.

[21]  Li Xia,et al.  A Structure Property of Optimal Policies for Maintenance Problems WithSafety-Critical Components , 2008, IEEE Transactions on Automation Science and Engineering.

[22]  Koushik Kar,et al.  Near-optimal activation policies in rechargeable sensor networks under spatial correlations , 2008, TOSN.

[23]  C. Pornpanomchai,et al.  Vehicle detection and counting from a video frame , 2008, 2008 International Conference on Wavelet Analysis and Pattern Recognition.

[24]  Tsong-Yi Chen,et al.  People Counting System for Getting In/Out of a Bus Based on Video Processing , 2008, 2008 Eighth International Conference on Intelligent Systems Design and Applications.

[25]  Qing-Shan Jia,et al.  An Indoor Localization Algorithm for Lighting Control using RFID , 2008, 2008 IEEE Energy 2030 Conference.

[26]  Xi Chen,et al.  Tracking moving object via a sensor network with partial information broadcasting scheme , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[27]  Qing-Shan Jia,et al.  Strategy optimization for controlled Markov process with descriptive complexity constraint , 2009, Science in China Series F: Information Sciences.

[28]  Xi Chen,et al.  Complete coverage and point coverage in randomly distributed sensor networks , 2009, Autom..

[29]  Qing-Shan Jia,et al.  Estimation of occupancy level in indoor environment based on heterogeneous information fusion , 2010, 49th IEEE Conference on Decision and Control (CDC).

[30]  Qing-Shan Jia.  On solving optimal policies for event-based dynamic programming , 2010, Proceedings of the 29th Chinese Control Conference.

[31]  Qing-Shan Jia.  A Structural Property of Optimal Policies for Multi-Component Maintenance Problems , 2010, IEEE Transactions on Automation Science and Engineering.

[32]  Qing-Shan Jia,et al.  On State Aggregation to Approximate Complex Value Functions in Large-Scale Markov Decision Processes , 2011, IEEE Transactions on Automatic Control.

[33]  Qing-Shan Jia,et al.  An Adaptive Sampling Algorithm for Simulation-Based Optimization With Descriptive Complexity Preference , 2011, IEEE Transactions on Automation Science and Engineering.

[34]  Ling Shi,et al.  On optimal partial broadcasting of wireless sensor networks for Kalman filtering , 2011, Proceedings of the 2011 American Control Conference.

[35]  Lei Huang,et al.  Robust people counting in video surveillance: Dataset and system , 2011, 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[36]  Qing-Shan Jia,et al.  On Solving Event-Based Optimization With Average Reward Over Infinite Stages , 2011, IEEE Transactions on Automatic Control.

[37]  Qing-Shan Jia,et al.  On Solving Optimal Policies for Finite-Stage Event-Based Optimization , 2011, IEEE Transactions on Automatic Control.

[38]  Jia Qing-Shan.  Event-based optimization with lagged state information , 2012, Proceedings of the 31st Chinese Control Conference.

[39]  Jia Qing-Shan,et al.  Event-based evacuation in outdoor environment , 2012, 2012 24th Chinese Control and Decision Conference (CCDC).

[40]  Steven I. Marcus,et al.  Simulation-based Algorithms for Markov Decision Processes/ Hyeong Soo Chang ... [et al.] , 2013 .