Optimizing Application Performance through Learning and Cooperation in a Wireless Sensor Network

A wireless sensor network performing surveillance in time-critical missions involving event or target tracking demands accurate ground information be delivered within a delay guarantee. Present methods solve this by using in-network fusion across all packets to reduce network load in the hope of achieving the delay guarantee. In this paper, we aim to maximize data quality from sensor fusion, while still respecting delay guarantees. The proposed method makes admission control and routing decisions using a fully distributed algorithm based on constrained Markov Decision Processes (MDPs). Cooperation is enforced through well-defined rewards and leading nodes. Assessment of data quality is derived from likelihood ratio, which is a commonly used metric in sensor fusion. We study the performance of the proposed algorithm through extensive simulations, and show that it can achieve soft delay guarantees and good data quality compared to other schemes.

[1]  Stephen P. Boyd,et al.  A scheme for robust distributed sensor fusion based on average consensus , 2005, IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005..

[2]  Pramod K. Varshney,et al.  A Bayesian sampling approach to decision fusion using hierarchical models , 2002, IEEE Trans. Signal Process..

[3]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[4]  Chen-Khong Tham,et al.  Distributed Model-Free Stochastic Optimization in Wireless Sensor Networks , 2006, DCOSS.

[5]  Chen-Khong Tham,et al.  Modular on-line function approximation for scaling up reinforcement learning , 1994 .

[6]  Martin Lauer,et al.  An Algorithm for Distributed Reinforcement Learning in Cooperative Multi-Agent Systems , 2000, ICML.

[7]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[8]  E. Altman Constrained Markov Decision Processes , 1999 .

[9]  Robert Nowak,et al.  Distributed optimization in sensor networks , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[10]  A. Varga,et al.  THE OMNET++ DISCRETE EVENT SIMULATION SYSTEM , 2003 .

[11]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[12]  Y. C. Tay,et al.  Collision-minimizing CSMA and its applications to wireless sensor networks , 2004, IEEE Journal on Selected Areas in Communications.

[13]  Chenyang Lu,et al.  SPEED: a stateless protocol for real-time communication in sensor networks , 2003, 23rd International Conference on Distributed Computing Systems, 2003. Proceedings..

[14]  Srinivasan Seshan,et al.  Synopsis diffusion for robust aggregation in sensor networks , 2004, SenSys '04.

[15]  Parameswaran Ramanathan,et al.  Distributed target classification and tracking in sensor networks , 2003 .