PERFORMANCE EVALUATION OF REAL-TIME EVENT DETECTION ALGORITHMS

The recent heightened awareness of the possibility of terrorist attacks on municipal water systems has prompted the development of the necessary hardware and software for implementing a Contamination Warning System (CWS) to detect if an attack has been made in a water distribution system. Much progress has been made in this area, including the development of improved water quality sensors, algorithms for placing the sensors most effectively in a water system, and algorithms for analyzing the data that comes from these sensors. This work focuses on this last component of analyzing sensor data. Several algorithms have been developed to tackle this problem. However, the various methods have never been tested by an independent party or compared to each other. This paper describes the preliminary work that has been done to design and implement a fair and thorough evaluation of algorithms to determine which is most effective in detecting a water contamination event.

[1]  Tom Fawcett,et al.  Activity monitoring: noticing interesting changes in behavior , 1999, KDD '99.

[2]  R.K. Cunningham,et al.  Evaluating intrusion detection systems: the 1998 DARPA off-line intrusion detection evaluation , 2000, Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00.

[3]  S. Pehkonen,et al.  Oxidation of diazinon by aqueous chlorine: kinetics, mechanisms, and product studies. , 1999, Journal of agricultural and food chemistry.

[4]  David E. Culler,et al.  Design of a wireless sensor network platform for detecting rare, random, and ephemeral events , 2005, IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005..

[5]  Mauricio G. C. Resende,et al.  A Hybrid Heuristic for the p-Median Problem , 2004, J. Heuristics.

[6]  Ian F. Akyildiz,et al.  Wireless sensor networks: a survey , 2002, Comput. Networks.

[7]  W.G. Zanardelli,et al.  Failure Prognosis for Permanent Magnet AC Drives Based on Wavelet Analysis , 2005, IEEE International Conference on Electric Machines and Drives, 2005..

[8]  Alvin F. Martin,et al.  The DET curve in assessment of detection task performance , 1997, EUROSPEECH.

[9]  Michael Berg,et al.  Kinetics and mechanistic aspects of As(III) oxidation by aqueous chlorine, chloramines, and ozone: relevance to drinking water treatment. , 2006, Environmental science & technology.

[10]  Avi Ostfeld,et al.  A Stochastic Early Warning Detection System Model for Drinking Water Distribution Systems Security , 2004 .

[11]  Jonathan W. Berry,et al.  Scalable High-Performance Heuristics for Sensor Placement in Water Distribution Networks , 2005 .

[12]  James G. Uber,et al.  Greedy Heuristic Methods for Locating Water Quality Sensors in Distribution Systems , 2004 .

[13]  Sang Hyuk Son,et al.  Event Detection Services Using Data Service Middleware in Distributed Sensor Networks , 2003, Telecommun. Syst..

[14]  Simin Nadjm-Tehrani,et al.  ADWICE - Anomaly Detection with Real-Time Incremental Clustering , 2004, ICISC.

[15]  L. Hutwagner,et al.  The bioterrorism preparedness and response Early Aberration Reporting System (EARS) , 2003, Journal of Urban Health.

[16]  David L Buckeridge,et al.  Evaluation of syndromic surveillance systems--design of an epidemic simulation model. , 2004, MMWR supplements.

[17]  Akbar M. Sayeed,et al.  Detection, Classification and Tracking of Targets in Distributed Sensor Networks , 2002 .

[18]  Ambuj K. Singh,et al.  Real-time nondestructive structural health monitoring using support vector machines and wavelets , 2005, SPIE Smart Structures and Materials + Nondestructive Evaluation and Health Monitoring.

[19]  Mark Gibson,et al.  Technologies and Techniques for Early Warning Systems to Monitor and Evaluate Drinking Water Quality: A State-of-the-Art Review , 2005 .

[20]  Avi Ostfeld,et al.  Optimal Layout of Early Warning Detection Stations for Water Distribution Systems Security , 2004 .

[21]  S. Sitharama Iyengar,et al.  Distributed Bayesian algorithms for fault-tolerant event region detection in wireless sensor networks , 2004, IEEE Transactions on Computers.

[22]  Satish Kumar,et al.  Next century challenges: scalable coordination in sensor networks , 1999, MobiCom.

[23]  Lorenz T. Biegler,et al.  A Mixed Integer Approach for Obtaining Unique Solutions in Source Inversion of Drinking Water Networks , 2005 .

[24]  D. Sosin.  Draft framework for evaluating syndromic surveillance systems , 2003, Journal of Urban Health.

[25]  James G. Uber,et al.  The Threat Ensemble Vulnerability Assessment (TEVA) Program for Drinking Water Distribution System Security , 2004 .

[26]  Sameer Singh,et al.  Novelty detection: a review - part 2: : neural network based approaches , 2003, Signal Process..

[27]  Cynthia A. Phillips,et al.  A General Integer-Programming-Based Framework for Sensor Placement in Municipal Water Networks , 2004 .

[28]  Lee M. Rossey,et al.  Extending the DARPA off-line intrusion detection evaluations , 2001, Proceedings DARPA Information Survivability Conference and Exposition II. DISCEX'01.

[29]  Richard Lippmann,et al.  The 1999 DARPA off-line intrusion detection evaluation , 2000, Comput. Networks.

[30]  Xin Chen,et al.  Analyzing Object Detection Quality Under Probabilistic Coverage in Sensor Networks , 2005, IWQoS.

[31]  Avi Ostfeld,et al.  An Early Warning Detection System (EWDS) for Drinking Water Distribution Systems Security , 2003 .

[32]  Jun Zhang,et al.  Detection of Outbreaks from Time Series Data Using Wavelet Transform , 2003, AMIA.

[33]  S. Blount,et al.  Lead Visual Information Specialist , 2003 .

[34]  Andrew W. Moore,et al.  Algorithms for rapid outbreak detection: a research synthesis , 2005, J. Biomed. Informatics.

[35]  Peter J. Bickel,et al.  A New Methodology for Evaluating Incident Detection Algorithms , 2000 .

[36]  J. Pavlin,et al.  Bio-ALIRT biosurveillance detection algorithm evaluation. , 2004, MMWR supplements.

[37]  Lorenz T. Biegler,et al.  Contamination Source Determination for Water Networks , 2005 .

[38]  Salvatore J. Stolfo,et al.  Distributed data mining in credit card fraud detection , 1999, IEEE Intell. Syst..

[39]  David S. Johnson,et al.  A theoretician's guide to the experimental analysis of algorithms , 1999, Data Structures, Near Neighbor Searches, and Methodology.

[40]  Young-Jin Kim,et al.  Multi-dimensional range queries in sensor networks , 2003, SenSys '03.