Fractal Based Anomaly Detection over Data Streams

Robust and efficient approaches are needed in real-time monitoring of data streams. In this paper, we focus on anomaly detection on data streams. Existing methods on anomaly detection suffer three problems. 1) A large volume of false positive results are generated. 2) The training data are needed, and the time window of appropriate size along with corresponding threshold has to be determined empirically. 3) Both time and space overhead is usually very high. We propose a novel self-similarity-based anomaly detection algorithm based on piecewise fractal model. This algorithm consumes only limited amount of memory and does not require training process. Theoretical analysis of the algorithm are presented. The experimental results on the real data sets indicate that, compared with existing anomaly detection methods, our algorithm can achieve higher precision with reduced space and time complexity.

[1]  Cyrus Shahabi,et al.  TSA-tree: a wavelet-based approach to improve the efficiency of multi-level surprise and trend queries on time-series data , 2000, Proceedings. 12th International Conference on Scientific and Statistica Database Management.

[2]  Michael F. Barnsley,et al.  Fractals everywhere , 1988 .

[3]  John C. Hart Fractal image compression and recurrent iterated function systems , 1996, IEEE Computer Graphics and Applications.

[4]  P.-O. Amblard,et al.  Stochastic discrete scale invariance , 2002, IEEE Signal Processing Letters.

[5]  Jung-Min Park,et al.  An overview of anomaly detection techniques: Existing solutions and latest technological trends , 2007, Comput. Networks.

[6]  Dennis Shasha,et al.  Efficient elastic burst detection in data streams , 2003, KDD '03.

[7]  M. Barnsley,et al.  Recurrent iterated function systems , 1989 .

[8]  Edward Y. Chang,et al.  Adaptive stream resource management using Kalman Filters , 2004, SIGMOD '04.

[9]  Graham Cormode,et al.  What's new: finding significant differences in network data streams , 2004, INFOCOM 2004.

[10]  Benoit B. Mandelbrot,et al.  Fractal Geometry of Nature , 1984 .

[11]  Ambuj K. Singh,et al.  A unified framework for monitoring data streams in real time , 2005, 21st International Conference on Data Engineering (ICDE'05).

[12]  Aoying Zhou,et al.  Approximately Processing Multi-granularity Aggregate Queries over Data Streams , 2006, 22nd International Conference on Data Engineering (ICDE'06).

[13]  Balachander Krishnamurthy,et al.  Sketch-based change detection: methods, evaluation, and applications , 2003, IMC '03.

[14]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[15]  Yixin Chen,et al.  Multi-Dimensional Regression Analysis of Time-Series Data Streams , 2002, VLDB.

[16]  Shai Ben-David,et al.  Detecting Change in Data Streams , 2004, VLDB.

[17]  Monson H. Hayes,et al.  Using iterated function systems to model discrete sequences , 1992, IEEE Trans. Signal Process..

[18]  Aoying Zhou,et al.  Adaptively Detecting Aggregation Bursts in Data Streams , 2005, DASFAA.