Novelty Detection Using Level Set Methods

This paper presents a level set boundary description (LSBD) approach for novelty detection that treats the nonlinear boundary directly in the input space. The proposed approach consists of level set function (LSF) construction, boundary evolution, and termination of the training process. It employs kernel density estimation to construct the LSF of the initial boundary for the training data set. Then, a sign of the LSF-based algorithm is proposed to evolve the boundary and make it fit more tightly in the data distribution. The training process terminates when an expected fraction of rejected normal data is reached. The evolution process utilizes the signs of the LSF values at all training data points to decide whether to expand or shrink the boundary. Extensive experiments are conducted on benchmark data sets to evaluate the proposed LSBD method and compare it against four representative novelty detection methods. The experimental results demonstrate that the novelty detector modeled with the proposed LSBD can effectively detect anomalies.

[1]  Tapani Raiko,et al.  Semi-supervised detection of collective anomalies with an application in high energy particle physics , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[2]  Bernhard Schölkopf,et al.  Support Vector Method for Novelty Detection , 1999, NIPS.

[3]  H. Bungartz,et al.  Sparse grids , 2004, Acta Numerica.

[4]  Fabrizio Angiulli,et al.  Prototype-Based Domain Description for One-Class Classification , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  James A. Sethian,et al.  Level Set Methods and Fast Marching Methods: Evolving Interfaces in Computational Geometry, Fluid , 2012 .

[6]  Charu C. Aggarwal,et al.  Outlier Detection for Temporal Data: A Survey , 2014, IEEE Transactions on Knowledge and Data Engineering.

[7]  Kush R. Varshney,et al.  Classification Using Geometric Level Sets , 2010, J. Mach. Learn. Res..

[8]  H. Lian.  On feature selection with principal component analysis for one-class SVM , 2012, Pattern Recognit. Lett..

[9]  Hanan Samet,et al.  K-Nearest Neighbor Finding Using MaxNearestDist , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  J.S.H. Tsai,et al.  A boundary method for outlier detection based on support vector domain description , 2009, Pattern Recognit..

[11]  Martin K. Purvis,et al.  Novelty detection in wildlife scenes through semantic context modelling , 2012, Pattern Recognit..

[12]  David A. Clifton,et al.  Novelty Detection for Identifying Deterioration in Emergency Department Patients , 2011, IDEAL.

[13]  Yuhua Li,et al.  Selecting training points for one-class support vector machines , 2011, Pattern Recognit. Lett..

[14]  Sameer Singh,et al.  Novelty detection: a review - part 1: statistical approaches , 2003, Signal Process..

[15]  Ronald Fedkiw,et al.  Level set methods and dynamic implicit surfaces , 2002, Applied mathematical sciences.

[16]  Markus Timusk,et al.  Feature extraction for novelty detection as applied to fault detection in machinery , 2011, Pattern Recognit. Lett..

[17]  Adriano Lorena Inácio de Oliveira,et al.  Combining nearest neighbor data description and structural risk minimization for one-class classification , 2009, Neural Computing and Applications.

[18]  David A. Clifton,et al.  Novelty Detection with Multivariate Extreme Value Statistics , 2011, J. Signal Process. Syst..

[19]  Victoria M Catterson,et al.  Online Conditional Anomaly Detection in Multivariate Data for Transformer Monitoring , 2010, IEEE Transactions on Power Delivery.

[20]  Robert P.W. Duin,et al.  PRTools3: A Matlab Toolbox for Pattern Recognition , 2000 .

[21]  Trung Le,et al.  An optimal sphere and two large margins approach for novelty detection , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[22]  Miguel Lázaro-Gredilla,et al.  Adaptive One-Class Support Vector Machine , 2011, IEEE Transactions on Signal Processing.

[23]  Xiongcai Cai,et al.  Level Learning Set: A Novel Classifier Based on Active Contour Models , 2007, ECML.

[24]  Korris Fu-Lai Chung,et al.  Theoretical analysis for solution of support vector data description , 2011, Neural Networks.

[25]  Gilles Blanchard,et al.  Semi-Supervised Novelty Detection , 2010, J. Mach. Learn. Res..

[26]  Sameer Singh,et al.  Novelty detection: a review - part 2: : neural network based approaches , 2003, Signal Process..

[27]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[28]  Ian M. Mitchell.  The Flexible, Extensible and Efficient Toolbox of Level Set Methods , 2008, J. Sci. Comput..

[29]  Adriano Lorena Inácio de Oliveira,et al.  A hybrid method for novelty detection in time series based on states transitions and swarm intelligence , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[30]  Bhavani M. Thuraisingham,et al.  Classification and Novel Class Detection in Concept-Drifting Data Streams under Time Constraints , 2011, IEEE Transactions on Knowledge and Data Engineering.

[31]  Venkatesh Saligrama,et al.  Abnormality detection using low-level co-occurring events , 2011, Pattern Recognit. Lett..

[32]  Bernhard Schölkopf,et al.  Support Vector Novelty Detection Applied to Jet Engine Vibration Spectra , 2000, NIPS.

[33]  Marius Kloft,et al.  Toward Supervised Anomaly Detection , 2014, J. Artif. Intell. Res..

[34]  Robert Sabourin,et al.  Iterative Boolean combination of classifiers in the ROC space: An application to anomaly detection with HMMs , 2010, Pattern Recognit..

[35]  Alex M. Andrew,et al.  Level Set Methods and Fast Marching Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Materials Science (2nd edition) , 2000 .

[36]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[37]  Nikos Fakotakis,et al.  Probabilistic Novelty Detection for Acoustic Surveillance Under Real-World Conditions , 2011, IEEE Transactions on Multimedia.

[38]  Tim Reiner,et al.  Interactive modeling of implicit surfaces using a direct visualization approach with signed distance functions , 2011, Comput. Graph..

[39]  Zhongyu WEI,et al.  One-class Classification based Finance News Story Recommendation , 2010 .

[40]  Wenkai Li,et al.  A Positive and Unlabeled Learning Algorithm for One-Class Classification of Remote-Sensing Data , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[41]  J. Sethian,et al.  Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations , 1988 .

[42]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[43]  Robert P. W. Duin,et al.  Minimum spanning tree based one-class classifier , 2009, Neurocomputing.

[44]  S. Osher,et al.  Algorithms Based on Hamilton-Jacobi Formulations , 1988 .

[45]  Arkadiusz Tomczyk,et al.  On the Relationship Between Active Contours and Contextual Classification , 2005, CORES.

[46]  Lionel Tarassenko,et al.  Static and dynamic novelty detection methods for jet engine health monitoring , 2007, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[47]  Xiaofeng Wang,et al.  A Novel Density-Based Clustering Framework by Using Level Set Method , 2009, IEEE Transactions on Knowledge and Data Engineering.

[48]  Bingru Yang,et al.  A new Outlier detection algorithm based on Manifold Learning , 2010, 2010 Chinese Control and Decision Conference.

[49]  Sankar K. Pal,et al.  Fuzzy sets and decisionmaking approaches in vowel and speaker recognition , 1977 .

[50]  Yixin Chen,et al.  Outlier Detection with the Kernelized Spatial Depth Function , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Daniel G. Sbarbaro-Hofer,et al.  Outliers detection in environmental monitoring databases , 2011, Eng. Appl. Artif. Intell..

[52]  Ratna Babu Chinnam,et al.  An Industrial Strength Novelty Detection Framework for Autonomous Equipment Monitoring and Diagnostics , 2010, IEEE Transactions on Industrial Informatics.

[53]  Marimuthu Palaniswami,et al.  Measures for clustering and anomaly detection in sets of higher dimensional ellipsoids , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[54]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[55]  Chandan Srivastava,et al.  Support Vector Data Description , 2011 .

[56]  Dae-Won Kim,et al.  Density-Induced Support Vector Data Description , 2007, IEEE Transactions on Neural Networks.

[57]  Robert D. Nowak,et al.  Minimax Optimal Level-Set Estimation , 2007, IEEE Transactions on Image Processing.

[58]  L. Tarassenko,et al.  Bayesian Extreme Value Statistics for Novelty Detection in Gas-Turbine Engines , 2008, 2008 IEEE Aerospace Conference.

[59]  Keith Worden,et al.  The use of pseudo-faults for novelty detection in SHM , 2010 .

[60]  Yu Ding,et al.  A Computable Plug-In Estimator of Minimum Volume Sets for Novelty Detection , 2010, Oper. Res..

[61]  Di Wu,et al.  Comparison of Binary Classification Based on Signed Distance Functions with Support Vector Machines , 2008, 2009 Ohio Collaborative Conference on Bioinformatics.

[62]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[63]  Jugal K. Kalita,et al.  A Survey of Outlier Detection Methods in Network Anomaly Identification , 2011, Comput. J..

[64]  Denis Friboulet,et al.  Compactly Supported Radial Basis Functions Based Collocation Method for Level-Set Evolution in Image Segmentation , 2007, IEEE Transactions on Image Processing.

[65]  Yi-Hung Liu,et al.  Fast Support Vector Data Descriptions for Novelty Detection , 2010, IEEE Transactions on Neural Networks.

[66]  Yuhua Li,et al.  Selecting Critical Patterns Based on Local Geometrical and Statistical Information , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[67]  Andreas Rauber,et al.  Decision Manifolds—A Supervised Learning Algorithm Based on Self-Organization , 2008, IEEE Transactions on Neural Networks.

[68]  Jon Louis Bentley,et al.  An Algorithm for Finding Best Matches in Logarithmic Expected Time , 1977, TOMS.

[69]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[70]  Cecilia Surace,et al.  Novelty detection in a changing environment: A negative selection approach , 2010 .

[71]  Marimuthu Palaniswami,et al.  Clustering ellipses for anomaly detection , 2011, Pattern Recognit..

[72]  Chris H. Q. Ding,et al.  Dynamic cluster formation using level set methods , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[73]  Dong Xu,et al.  Efficient support vector data descriptions for novelty detection , 2011, Neural Computing and Applications.

[74]  Le Song,et al.  Relative Novelty Detection , 2009, AISTATS.

[75]  Jiří Zelinka,et al.  Kernel Density Estimation Toolbox for Matlab , 2011 .

[76]  Jung-Min Park,et al.  An overview of anomaly detection techniques: Existing solutions and latest technological trends , 2007, Comput. Networks.

[77]  Victoria J. Hodge,et al.  A Survey of Outlier Detection Methodologies , 2004, Artificial Intelligence Review.

[78]  Guido Sanguinetti,et al.  Information theoretic novelty detection , 2010, Pattern Recognit..

[79]  Jieping Ye,et al.  A Small Sphere and Large Margin Approach for Novelty Detection Using Training Data with Outliers , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[80]  Daniel J. Strauss,et al.  Feasibility of an objective electrophysiological loudness scaling: A kernel-based novelty detection approach , 2012, Artif. Intell. Medicine.

[81]  Luigi Palopoli,et al.  Outlier detection for simple default theories , 2010, Artif. Intell..

[82]  Cungen Cao,et al.  A hybrid approach to outlier detection based on boundary region , 2011, Pattern Recognit. Lett..