Robust Neural Network for Novelty Detection on Data Streams

In the on-line data processing it is important to detect a novelty as soon as it appears, because it may be a consequence of gross errors or sudden change in the analysed system. In this paper we present a framework of novelty detection, based on the robust neural network. To detect novel patterns we compare responses of two autoregressive neural networks. One of them is trained with a robust learning algorithm designed to remove the influence of outliers, while the other uses simple training, based on the least squares error criterion. We present also a simple and easy to use approach that adapts this technique to data streams. Experiments conducted on data containing novelty and outliers have shown promising performance of the new method, applied to analyse temporal sequences.

[1]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[2]  Robert K. Goodrich,et al.  An Algorithm for Classification and Outlier Detection of Time-Series Data , 2010 .

[3]  Junshui Ma,et al.  Online novelty detection on temporal sequences , 2003, KDD '03.

[4]  Carlo Zaniolo,et al.  An adaptive learning approach for noisy data streams , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[5]  Andrzej Rusiecki.  Fast Robust Learning Algorithm Dedicated to LMLS Criterion , 2010, ICAISC.

[6]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[7]  Paul A. Crook,et al.  A Robot Implementation of a Biologically Inspired Method for Novelty Detection , 2002 .

[8]  Andrzej Rusiecki,et al.  Robust MCD-Based Backpropagation Learning Algorithm , 2006, ICAISC.

[9]  Jacek M. Zurada,et al.  Artificial Intelligence and Soft Computing, 10th International Conference, ICAISC 2010, Zakopane, Poland, June 13-17, 2010, Part I , 2010, International Conference on Artificial Intelligence and Soft Computing.

[10]  Shun-Feng Su,et al.  The annealing robust backpropagation (ARBP) learning algorithm , 2000, IEEE Trans. Neural Networks Learn. Syst..

[11]  M.Kleinberg Jon,et al.  Advances in Self-Organizing Maps, 7th International Workshop, WSOM 2009, St. Augustine, FL, USA, June 8-10, 2009. Proceedings , 2009, WSOM.

[12]  T. Brotherton,et al.  Classification and novelty detection using linear models and a class dependent-elliptical basis function neural network , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[13]  Leonardo Aguayo,et al.  Time Series Clustering for Anomaly Detection Using Competitive Neural Networks , 2009, WSOM.

[14]  Jacek M. Zurada,et al.  Artificial Intelligence and Soft Computing - ICAISC 2008, 9th International Conference, Zakopane, Poland, June 22-26, 2008, Proceedings , 2008, ICAISC.

[15]  Ramesh C. Jain,et al.  A robust backpropagation learning algorithm for function approximation , 1994, IEEE Trans. Neural Networks.

[16]  Kadir Liano,et al.  Robust error measure for supervised neural network learning with outliers , 1996, IEEE Trans. Neural Networks.

[17]  Victoria J. Hodge,et al.  A Survey of Outlier Detection Methodologies , 2004, Artificial Intelligence Review.

[18]  Ana González-Marcos,et al.  TAO-robust backpropagation learning algorithm , 2005, Neural Networks.

[19]  O. Simula,et al.  The Self-organizing map as a tool in knowledge engineering , 2000 .

[20]  Paul D. Gader,et al.  Neural networks with enhanced outlier rejection ability for off-line handwritten word recognition , 2002, Pattern Recognit..

[21]  Hongxing He,et al.  Outlier Detection Using Replicator Neural Networks , 2002, DaWaK.