Novelty detection by multivariate kernel density estimation and growing neural gas algorithm

Abstract One of the underlying assumptions when using data-based methods for pattern recognition in diagnostics or prognostics is that the selected data sample used to train and test the algorithm is representative of the entire dataset and covers all combinations of parameters and conditions, and resulting system states. However in practice, operating and environmental conditions may change, unexpected and previously unanticipated events may occur and corresponding new anomalous patterns develop. Therefore for practical applications, techniques are required to detect novelties in patterns and give confidence to the user on the validity of the performed diagnosis and predictions. In this paper, the application of two types of novelty detection approaches is compared: a statistical approach based on multivariate kernel density estimation and an approach based on a type of unsupervised artificial neural network, called the growing neural gas (GNG). The comparison is performed on a case study in the field of railway turnout systems. Both approaches demonstrate their suitability for detecting novel patterns. Furthermore, GNG proves to be more flexible, especially with respect to dimensionality of the input data and suitability for online learning.

[1]  E. M. Wright,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[2]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[3]  Qinyu. Zhu.  Extreme Learning Machine , 2013 .

[4]  Daming Lin,et al.  A review on machinery diagnostics and prognostics implementing condition-based maintenance , 2006 .

[5]  Stelios Psarakis,et al.  Multivariate statistical process control charts: an overview , 2007, Qual. Reliab. Eng. Int..

[6]  Sameer Singh,et al.  Novelty detection: a review - part 1: statistical approaches , 2003, Signal Process..

[7]  Cecilia Surace,et al.  Novelty detection in a changing environment: A negative selection approach , 2010 .

[8]  Theodora Kourti,et al.  Statistical Process Control of Multivariate Processes , 1994 .

[9]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[10]  Sameer Singh,et al.  Novelty detection: a review - part 2: : neural network based approaches , 2003, Signal Process..

[11]  Asoke K. Nandi,et al.  Modified self-organising map for automated novelty detection applied to vibration signal monitoring , 2006 .

[12]  Hugh M. Cartwright.  Growing Cell Structures , 2008 .

[13]  Dit-Yan Yeung,et al.  Parzen-window network intrusion detectors , 2002, Object recognition supported by user interaction for service robots.

[14]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[15]  I. Jolliffe.  Principal Component Analysis , 2002 .

[16]  Bernd Fritzke,et al.  Fast learning with incremental RBF networks , 1994, Neural Processing Letters.

[17]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[18]  P. S. Heyns,et al.  Combining synchronous averaging with a Gaussian mixture model novelty detection scheme for vibration-based condition monitoring of a gearbox , 2012 .

[19]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[20]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[21]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[22]  R. Tibshirani,et al.  An introduction to the bootstrap , 1993 .

[23]  Stephen R. Marsland,et al.  On-line novelty detection for autonomous mobile robots , 2005, Robotics Auton. Syst..

[24]  James Hensman,et al.  Natural computing for mechanical systems research: A tutorial overview , 2011 .

[25]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[26]  W. Härdle.  Nonparametric and Semiparametric Models , 2004 .

[27]  A. E. Hoerl,et al.  Ridge regression: biased estimation for nonorthogonal problems , 2000 .

[28]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[29]  D. Dasgupta.  Artificial Immune Systems and Their Applications , 1998, Springer Berlin Heidelberg.