Entropic One-Class Classifiers

The one-class classification problem is a well-known research endeavor in pattern recognition. The problem is also known under different names, such as outlier and novelty/anomaly detection. The core of the problem consists in modeling and recognizing patterns belonging only to a so-called target class. All other patterns are termed nontarget, and therefore, they should be recognized as such. In this paper, we propose a novel one-class classification system that is based on an interplay of different techniques. Primarily, we follow a dissimilarity representation-based approach; we embed the input data into the dissimilarity space (DS) by means of an appropriate parametric dissimilarity measure. This step allows us to process virtually any type of data. The dissimilarity vectors are then represented by weighted Euclidean graphs, which we use to determine the entropy of the data distribution in the DS and at the same time to derive effective decision regions that are modeled as clusters of vertices. Since the dissimilarity measure for the input data is parametric, we optimize its parameters by means of a global optimization scheme, which considers both mesoscopic and structural characteristics of the data represented through the graphs. The proposed one-class classifier is designed to provide both hard (Boolean) and soft decisions about the recognition of test patterns, allowing an accurate description of the classification process. We evaluate the performance of the system on different benchmarking data sets, containing either feature-based or structured patterns. Experimental results demonstrate the effectiveness of the proposed technique.

[1]  Ying Xu,et al.  Clustering gene expression data using a graph-theoretic approach: an application of minimum spanning trees , 2002, Bioinform..

[2]  Marcello Pelillo,et al.  A Game-Theoretic Approach to Hypergraph Clustering , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Miguel Cazorla,et al.  Feature selection, mutual information, and the classification of high-dimensional patterns , 2008, Pattern Analysis and Applications.

[4]  Alfred O. Hero,et al.  Graph based k-means clustering , 2012, Signal Process..

[5]  Robert P. W. Duin,et al.  The Dissimilarity Representation for Pattern Recognition - Foundations and Applications , 2005, Series in Machine Perception and Artificial Intelligence.

[6]  Dan Stowell,et al.  Fast Multidimensional Entropy Estimation by $k$-d Partitioning , 2009, IEEE Signal Processing Letters.

[7]  Barnabás Póczos,et al.  Estimation of Renyi Entropy and Mutual Information Based on Generalized Nearest-Neighbor Graphs , 2010, NIPS.

[8]  Lorenzo Livi,et al.  Designing Labeled Graph Classifiers by Exploiting the Rényi Entropy of the Dissimilarity Representation , 2014, Entropy.

[9]  Alfred O. Hero,et al.  Asymptotic theory of greedy approximations to minimal k-point random graphs , 1999, IEEE Trans. Inf. Theory.

[10]  Mário A. T. Figueiredo,et al.  Soft clustering using weighted one-class support vector machines , 2009, Pattern Recognit..

[11]  Malik Yousef,et al.  One-Class SVMs for Document Classification , 2002, J. Mach. Learn. Res..

[12]  Ulrik Brandes,et al.  Experiments on Graph Clustering Algorithms , 2003, ESA.

[13]  Ammar Belatreche,et al.  An experimental evaluation of novelty detection methods , 2014, Neurocomputing.

[14]  Fabrizio Angiulli,et al.  Prototype-Based Domain Description for One-Class Classification , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Lorenzo Livi,et al.  The graph matching problem , 2012, Pattern Analysis and Applications.

[16]  Giosuè Lo Bosco,et al.  A Fuzzy One Class Classifier for Multi Layer Model , 2009, WILF.

[17]  Joachim Denzler,et al.  One-class classification with Gaussian processes , 2013, Pattern Recognit..

[18]  Robert P. W. Duin,et al.  The dissimilarity space: Bridging structural and statistical pattern recognition , 2012, Pattern Recognit. Lett..

[19]  Alfred O. Hero,et al.  Image matching using alpha-entropy measures and entropic graphs , 2005, Signal Process..

[20]  W. Pedrycz,et al.  An introduction to fuzzy sets : analysis and design , 1998 .

[21]  Alex Arenas,et al.  Analysis of the structure of complex networks at different resolution levels , 2007, physics/0703218.

[22]  Lorenzo Livi,et al.  Fault recognition in smart grids by a one-class classification approach , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[23]  David A. Clifton,et al.  A review of novelty detection , 2014, Signal Process..

[24]  Caroline Petitjean,et al.  A Random Forest Based Approach for One Class Classification in Medical Imaging , 2012, MLMI.

[25]  Witold Pedrycz,et al.  Anomaly Detection and Characterization in Spatial Time Series Data: A Cluster-Centric Approach , 2014, IEEE Transactions on Fuzzy Systems.

[26]  Ulrik Brandes,et al.  On Modularity Clustering , 2008, IEEE Transactions on Knowledge and Data Engineering.

[27]  Kaspar Riesen,et al.  IAM Graph Database Repository for Graph Based Pattern Recognition and Machine Learning , 2008, SSPR/SPR.

[28]  E. Rodner,et al.  Automatic identification of novel bacteria using Raman spectroscopy and Gaussian processes. , 2013, Analytica chimica acta.

[29]  Lorenzo Livi,et al.  Optimized dissimilarity space embedding for labeled graphs , 2014, Inf. Sci..

[30]  Lorenzo Livi,et al.  Graph ambiguity , 2013, Fuzzy Sets Syst..

[31]  Ludmila I. Kuncheva,et al.  Fuzzy Classifier Design , 2000, Studies in Fuzziness and Soft Computing.

[32]  Michael J. Laszlo,et al.  Minimum spanning tree partitioning algorithm for microaggregation , 2005, IEEE Transactions on Knowledge and Data Engineering.

[33]  Robert P. W. Duin,et al.  Minimum spanning tree based one-class classifier , 2009, Neurocomputing.

[34]  Gian Luca Foresti,et al.  Trajectory-Based Anomalous Event Detection , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[35]  Lorenzo Livi,et al.  Modeling and recognition of smart grid faults by a combined approach of dissimilarity learning and one-class classification , 2014, Neurocomputing.

[36]  Francisco Escolano,et al.  Information-theoretic selection of high-dimensional spectral features for structural recognition , 2013, Comput. Vis. Image Underst..

[37]  Francisco Herrera,et al.  A study on the use of statistical tests for experimentation with neural networks: Analysis of parametric test conditions and non-parametric tests , 2007, Expert Syst. Appl..

[38]  Alfred O. Hero,et al.  Determining Intrinsic Dimension and Entropy of High-Dimensional Shape Spaces , 2006, Statistics and Analysis of Shapes.

[39]  Alfred O. Hero,et al.  Applications of entropic spanning graphs , 2002, IEEE Signal Process. Mag..

[40]  Robert P. W. Duin,et al.  One-Class LP Classifiers for Dissimilarity Representations , 2002, NIPS.

[41]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[42]  Joachim Denzler,et al.  Kernel Null Space Methods for Novelty Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Robert P. W. Duin,et al.  Uniform Object Generation for Optimizing One-class Classifiers , 2002, J. Mach. Learn. Res..

[44]  Joachim Denzler,et al.  Divergence-Based One-Class Classification Using Gaussian Processes , 2012, BMVC.

[45]  Alfred O. Hero,et al.  Clustering with a new distance measure based on a dual-rooted tree , 2013, Inf. Sci..

[46]  Antonello Rizzi,et al.  Adaptive resolution min-max classifiers , 2002, IEEE Trans. Neural Networks.

[47]  Witold Pedrycz,et al.  Fuzzy sets in pattern recognition: Methodology and methods , 1990, Pattern Recognit..

[48]  Caroline Petitjean,et al.  One class random forests , 2013, Pattern Recognit..

[49]  Robert P. W. Duin,et al.  Support vector domain description , 1999, Pattern Recognit. Lett..

[50]  Chang-Dong Wang,et al.  Position regularized Support Vector Domain Description , 2013, Pattern Recognit..

[51]  Bartosz Krawczyk,et al.  On diversity measures for fuzzy one-class classifier ensembles , 2013, 2013 IEEE Symposium on Computational Intelligence and Ensemble Learning (CIEL).

[52]  Martin Rosvall,et al.  An information-theoretic framework for resolving community structure in complex networks , 2007, Proceedings of the National Academy of Sciences.

[53]  Bernhard Schölkopf,et al.  Support Vector Method for Novelty Detection , 1999, NIPS.

[54]  E. Ziv,et al.  Information-theoretic approach to network modularity. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[55]  Shehroz S. Khan,et al.  A Survey of Recent Trends in One Class Classification , 2009, AICS.

[56]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[57]  Lorenzo Livi,et al.  Dissimilarity space embedding of labeled graphs by a clustering-based compression procedure , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[58]  Giorgio Giacinto,et al.  One-class classification for oil spill detection , 2010, Pattern Analysis and Applications.

[59]  Lorenzo Livi,et al.  Building pattern recognition applications with the SPARE library , 2014, ArXiv.

[60]  Lev V. Utkin,et al.  Fuzzy One-Class Classification Model Using Contamination Neighborhoods , 2012, Adv. Fuzzy Syst..

[61]  Dae-Won Kim,et al.  Density-Induced Support Vector Data Description , 2007, IEEE Transactions on Neural Networks.

[62]  Loic Boisrobert,et al.  Assessment of artery dilation by using image registration based on spatial features , 2005, SPIE Medical Imaging.