On diversity measures for fuzzy one-class classifier ensembles

One-class classification became one of the most challenging research areas of the contemporary machine learning. Contrary to canonical task here we have only information about a single class at our disposal. Therefore more sophisticated methodologies, that are able to handle all the nuisances of the target distribution are required. Fuzzy logic seems an attractive solution to handle imprecision and to naturally grade the influence of the input data on the decision boundary. In this paper we propose to create a committee of fuzzy one-class support vector machines based on the random subspace method and diversity-based ensemble pruning technique. We investigate if there is a difference when using crisp or fuzzy diversity measures - and if so, then which of them are preferable for fuzzy one-class ensembles. The experimental investigations carried on a wide selection of benchmark datasets and backed-up with a statistical test of significance proves that selecting a proper diversity measure for fuzzy one-class ensemble has a crucial impact on its overall quality.

[1]  Piotr Juszczak.  Learning to recognise : a study on one-class classification and active learning , 2006 .

[2]  Ludmila I. Kuncheva,et al.  Clustering-and-selection model for classifier combination , 2000, KES'2000. Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies. Proceedings (Cat. No.00TH8516).

[3]  D. Tax,et al.  Characterizing one-class datasets , 2006 .

[4]  Bartosz Krawczyk,et al.  Diversity in Ensembles for One-Class Classification , 2012, ADBIS Workshops.

[5]  Michal Wozniak,et al.  Complexity and Multithreaded Implementation Analysis of One Class-Classifiers Fuzzy Combiner , 2011, HAIS.

[6]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[7]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[8]  Fabio Roli,et al.  Intrusion detection in computer networks by a modular ensemble of one-class classifiers , 2008, Inf. Fusion.

[9]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Lev V. Utkin,et al.  Fuzzy One-Class Classification Model Using Contamination Neighborhoods , 2012, Adv. Fuzzy Syst..

[11]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[12]  Bartosz Krawczyk,et al.  Experiments on distance measures for combining one-class classifiers , 2012, 2012 Federated Conference on Computer Science and Information Systems (FedCSIS).

[13]  Ethem Alpaydın,et al.  Combined 5 x 2 cv F Test for Comparing Supervised Classification Learning Algorithms , 1999, Neural Comput..

[14]  Michal Wozniak,et al.  Soft computing methods applied to combination of one-class classifiers , 2012, Neurocomputing.

[15]  Pei-Yi Hao,et al.  Fuzzy one-class support vector machines , 2008, Fuzzy Sets Syst..

[16]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[17]  Christopher M. Bishop,et al.  Novelty detection and neural network validation , 1994 .

[18]  Bartosz Krawczyk,et al.  Combining Diverse One-Class Classifiers , 2012, HAIS.

[19]  Giosuè Lo Bosco,et al.  A Fuzzy One Class Classifier for Multi Layer Model , 2009, WILF.

[20]  C. J. Whitaker,et al.  Ten measures of diversity in classifier ensembles: limits for two classifiers , 2001 .

[21]  Robert P. W. Duin,et al.  Combining One-Class Classifiers , 2001, Multiple Classifier Systems.

[22]  M. M. Moya,et al.  Cueing, feature discovery, and one-class learning for synthetic aperture radar automatic target recognition , 1995, Neural Networks.

[23]  Emilio Corchado,et al.  Hybrid intelligent algorithms and applications , 2010, Inf. Sci..

[24]  Hisao Ishibuchi,et al.  Performance evaluation of fuzzy classifier systems for multidimensional pattern classification problems , 1999, IEEE Trans. Syst. Man Cybern. Part B.