A fresh perspective: Learning to sparsify for detection in massive noisy sensor networks

Can one trade sensor quality for quantity? While larger networks with greater sensor density promise to allow us to use noisier sensors yet measure subtler phenomena, aggregating data and designing decision rules is challenging. Motivated by dense, participatory seismic networks, we seek efficient aggregation methods for event detection. We propose to perform aggregation by sparsification: roughly, a sparsifying basis is a linear transformation that aggregates measurements from groups of sensors that tend to co-activate, and each event is observed by only a few groups of sensors. We show how a simple class of sparsifying bases provably improves detection with noisy binary sensors, even when only qualitative information about the network is available. We then describe how detection can be further improved by learning a better sparsifying basis from network observations or simulations. Learning can be done offline, and makes use of powerful off-the-shelf optimization packages. Our approach outperforms state of the art detectors on real measurements from seismic networks with hundreds of sensors, and on simulated epidemics in the Gnutella P2P communication network.

[1]  Raghu K. Ganti,et al.  Analysis of Data from a Taxi Cab Participatory Sensor Network , 2011, MobiQuitous.

[2]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[3]  Andrew W. Moore,et al.  Rapid detection of significant spatial clusters , 2004, KDD.

[4]  David Bruce Wilson,et al.  Generating random spanning trees more quickly than the cover time , 1996, STOC '96.

[5]  Michael Elad,et al.  K-SVD : DESIGN OF DICTIONARIES FOR SPARSE REPRESENTATION , 2005 .

[6]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[7]  Quan Wang,et al.  Regularized latent semantic indexing , 2011, SIGIR.

[8]  Wei Hong,et al.  TinyDB: an acquisitional query processing system for sensor networks , 2005, TODS.

[9]  Ann B. Lee,et al.  Treelets--An adaptive multi-scale basis for sparse unordered data , 2007, 0707.0481.

[10]  ten Josephus Berge,et al.  Review of: J.C. Gower & G.B. Dijksterhuis: Procrustes Problems, Oxford University Press. , 2004 .

[11]  J. C. Gower,et al.  Projection Procrustes problems , 2004 .

[12]  J. Tsitsiklis Decentralized Detection' , 1993 .

[13]  E. Candès,et al.  Detection of an anomalous cluster in a network , 2010, 1001.3209.

[14]  Howard S. Burkom,et al.  Statistical Challenges Facing Early Outbreak Detection in Biosurveillance , 2010, Technometrics.

[15]  K. Mani Chandy,et al.  Towards a discipline of geospatial distributed event based systems , 2012, DEBS.

[16]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[17]  Quoc V. Le,et al.  ICA with Reconstruction Cost for Efficient Overcomplete Feature Learning , 2011, NIPS.

[18]  Leonidas J. Guibas,et al.  Sparse Data Aggregation in Sensor Networks , 2007, 2007 6th International Symposium on Information Processing in Sensor Networks.

[19]  M. Kulldorff A spatial scan statistic , 1997 .

[20]  Andreas Krause,et al.  Community Seismic Network , 2012 .

[21]  Ronald R. Coifman,et al.  Multiscale Wavelets on Trees, Graphs and High Dimensional Data: Theory and Applications to Semi Supervised Learning , 2010, ICML.

[22]  Sara van de Geer,et al.  Statistics for High-Dimensional Data , 2011 .

[23]  Andreas Krause,et al.  The next big one: Detecting earthquakes and other rare events from community-based sensors , 2011, Proceedings of the 10th ACM/IEEE International Conference on Information Processing in Sensor Networks.

[24]  Xi Chen,et al.  Sparse Latent Semantic Analysis , 2011, SDM.

[25]  Akshay Krishnamurthy,et al.  Detecting Activations over Graphs using Spanning Tree Wavelet Bases , 2012, AISTATS.

[26]  Christopher Leckie,et al.  A survey of coordinated attacks and collaborative intrusion detection , 2010, Comput. Secur..

[27]  Daniel B. Neill,et al.  Fast subset scan for spatial pattern detection , 2012 .

[28]  A. Robert Calderbank,et al.  Detecting Weak but Hierarchically-Structured Patterns in Networks , 2010, AISTATS.

[29]  Sara van de Geer,et al.  Statistics for High-Dimensional Data: Methods, Theory and Applications , 2011 .