A Unifying Method for Outlier and Change Detection from Data Streams

Detection of outliers and identification of change points in a data stream are two very exciting topics in the area of data mining. This paper explores the relationship between these two issues, and presents a unifying method for dealing with both of them. This approach is based on a probabilistic model of time series whose parameters are updated adaptively. The forward and backward prediction errors over a sliding window are used to represent the deviation extent of an outlier and the change degree of a change point. Unlike former approaches, the present one uses fuzzy partition method and fuzzy decision principle to alarm possible outliers and changes, which is more appropriate for online and interactive data mining from data streams. Simulation results confirm the effectiveness of the proposed method

[1]  Johannes Gehrke,et al.  Querying and mining data streams: you only get one look a tutorial , 2002, SIGMOD '02.

[2]  Jianqing Fan,et al.  Nonlinear Time Series : Nonparametric and Parametric Methods , 2005 .

[3]  Kenji Yamanishi,et al.  Discovering outlier filtering rules from unlabeled data , 2001, KDD 2001.

[4]  H. D. Cheng,et al.  Thresholding using two-dimensional histogram and fuzzy entropy principle , 2000, IEEE Trans. Image Process..

[5]  Anupam Joshi,et al.  Fuzzy clustering for intrusion detection , 2003, The 12th IEEE International Conference on Fuzzy Systems, 2003. FUZZ '03..

[6]  Jaideep Srivastava,et al.  Event detection from time series data , 1999, KDD '99.

[7]  Zhi Li,et al.  A Unifying Method for Outlier and Change Detection from Data Streams , 2006 .

[8]  Denis Kouame,et al.  Adaptive AR and Neurofuzzy Approaches: Access to Cerebral Particle Signatures , 2006, IEEE Transactions on Information Technology in Biomedicine.

[9]  Philip S. Yu,et al.  Online Mining of Changes from Data Streams: Research Problems and Preliminary Results , 2003 .

[10]  Akira Maeda,et al.  Unsupervised Outlier Detection in Time Series Data , 2006, 22nd International Conference on Data Engineering Workshops (ICDEW'06).

[11]  Kenji Yamanishi,et al.  A unifying framework for detecting outliers and change points from time series , 2006, IEEE Transactions on Knowledge and Data Engineering.

[12]  Svetha Venkatesh,et al.  Activity recognition and abnormality detection with the switching hidden semi-Markov model , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[13]  Maria Luiza F. Velloso,et al.  Unsupervised change detection using fuzzy entropy principle , 2004, IGARSS 2004. 2004 IEEE International Geoscience and Remote Sensing Symposium.