A fresh perspective: Learning to sparsify for detection in massive noisy sensor networks

Can one trade sensor quality for quantity? While larger networks with greater sensor density promise to allow us to use noisier sensors yet measure subtler phenomena, aggregating data and designing decision rules is challenging. Motivated by dense, participatory seismic networks, we seek efficient aggregation methods for event detection. We propose to perform aggregation by sparsification: roughly, a sparsifying basis is a linear transformation that aggregates measurements from groups of sensors that tend to co-activate, and each event is observed by only a few groups of sensors. We show how a simple class of sparsifying bases provably improves detection with noisy binary sensors, even when only qualitative information about the network is available. We then describe how detection can be further improved by learning a better sparsifying basis from network observations or simulations. Learning can be done offline, and makes use of powerful off-the-shelf optimization packages. Our approach outperforms state of the art detectors on real measurements from seismic networks with hundreds of sensors, and on simulated epidemics in the Gnutella P2P communication network.

[1]  David Bruce Wilson,et al.  Generating random spanning trees more quickly than the cover time , 1996, STOC '96.

[2]  J. Tsitsiklis Decentralized Detection' , 1993 .

[3]  Ronald R. Coifman,et al.  Multiscale Wavelets on Trees, Graphs and High Dimensional Data: Theory and Applications to Semi Supervised Learning , 2010, ICML.

[4]  Michael Elad,et al.  K-SVD : DESIGN OF DICTIONARIES FOR SPARSE REPRESENTATION , 2005 .

[5]  A. Robert Calderbank,et al.  Detecting Weak but Hierarchically-Structured Patterns in Networks , 2010, AISTATS.

[6]  Akshay Krishnamurthy,et al.  Detecting Activations over Graphs using Spanning Tree Wavelet Bases , 2012, AISTATS.

[7]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[8]  Ann B. Lee,et al.  Treelets--An adaptive multi-scale basis for sparse unordered data , 2007, 0707.0481.

[9]  Andrew W. Moore,et al.  Rapid detection of significant spatial clusters , 2004, KDD.

[10]  Quoc V. Le,et al.  ICA with Reconstruction Cost for Efficient Overcomplete Feature Learning , 2011, NIPS.

[11]  Sara van de Geer,et al.  Statistics for High-Dimensional Data , 2011 .

[12]  Wei Hong,et al.  TinyDB: an acquisitional query processing system for sensor networks , 2005, TODS.

[13]  R. Varga,et al.  Proof of Theorem 2 , 1983 .

[14]  Raghu K. Ganti,et al.  Analysis of Data from a Taxi Cab Participatory Sensor Network , 2011, MobiQuitous.

[15]  Christopher Leckie,et al.  A survey of coordinated attacks and collaborative intrusion detection , 2010, Comput. Secur..

[16]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[17]  Andreas Krause,et al.  Community Seismic Network , 2012 .

[18]  Daniel B. Neill,et al.  Fast subset scan for spatial pattern detection , 2012 .

[19]  J. C. Gower,et al.  Projection Procrustes problems , 2004 .

[20]  ten Josephus Berge,et al.  Review of: J.C. Gower & G.B. Dijksterhuis: Procrustes Problems, Oxford University Press. , 2004 .

[21]  Xi Chen,et al.  Sparse Latent Semantic Analysis , 2011, SDM.

[22]  Andreas Krause,et al.  The next big one: Detecting earthquakes and other rare events from community-based sensors , 2011, Proceedings of the 10th ACM/IEEE International Conference on Information Processing in Sensor Networks.

[23]  Howard S. Burkom,et al.  Statistical Challenges Facing Early Outbreak Detection in Biosurveillance , 2010, Technometrics.

[24]  Sara van de Geer,et al.  Statistics for High-Dimensional Data: Methods, Theory and Applications , 2011 .

[25]  M. Kulldorff A spatial scan statistic , 1997 .

[26]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[27]  E. Candès,et al.  Detection of an anomalous cluster in a network , 2010, 1001.3209.

[28]  Quan Wang,et al.  Regularized latent semantic indexing , 2011, SIGIR.

[29]  K. Mani Chandy,et al.  Towards a discipline of geospatial distributed event based systems , 2012, DEBS.