One-Class Classification with Extreme Learning Machine

One-class classification problem has been investigated thoroughly for past decades. Among one of the most effective neural network approaches for one-class classification, autoencoder has been successfully applied for many applications. However, this classifier relies on traditional learning algorithms such as backpropagation to train the network, which is quite time-consuming. To tackle the slow learning speed in autoencoder neural network, we propose a simple and efficient one-class classifier based on extreme learning machine (ELM). The essence of ELM is that the hidden layer need not be tuned and the output weights can be analytically determined, which leads to much faster learning speed. The experimental evaluation conducted on several real-world benchmarks shows that the ELM based one-class classifier can learn hundreds of times faster than autoencoder and it is competitive over a variety of one-class classification methods.

[1]  Michael I. Jordan,et al.  Robust Novelty Detection with Single-Class MPM , 2002, NIPS.

[2]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.

[3]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[4]  J. Platt Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[5]  J. Mendel Uncertain Rule-Based Fuzzy Logic Systems: Introduction and New Directions , 2001 .

[6]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[7]  Robert P. W. Duin,et al.  Minimum spanning tree based one-class classifier , 2009, Neurocomputing.

[8]  Yuan Lan,et al.  Ensemble of online sequential extreme learning machine , 2009, Neurocomputing.

[9]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[10]  Brian Mac Namee,et al.  Credit Scoring: Solving the Low Default Portfolio Problem Using One-Class Classification , 2009 .

[11]  Chee Kheong Siew,et al.  Incremental extreme learning machine with fully complex hidden nodes , 2008, Neurocomputing.

[12]  Zhan-Li Sun,et al.  A Neuro-Fuzzy Inference System Through Integration of Fuzzy Logic and Extreme Learning Machines , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[13]  Jun Miao,et al.  Constrained Extreme Learning Machine: A novel highly discriminative random feedforward neural network , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[14]  A. G. Arkad'ev,et al.  Computers and pattern recognition , 1967 .

[15]  Klaus-Robert Müller,et al.  Incremental Support Vector Learning: Analysis, Implementation and Applications , 2006, J. Mach. Learn. Res..

[16]  Shehroz S. Khan,et al.  One-class classification: taxonomy of study and review of techniques , 2013, The Knowledge Engineering Review.

[17]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[18]  Raymond T. Ng,et al.  Distance-based outliers: algorithms and applications , 2000, The VLDB Journal.

[19]  Antoine Geissbühler,et al.  An Application of One-class Support Vector Machines to Nosocomial Infection Detection , 2004, MedInfo.

[20]  David M. J. Tax,et al.  One-class classification , 2001 .

[21]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[22]  T. M. Williams Practical Methods of Optimization. Vol. 2 — Constrained Optimization , 1982 .

[23]  Robert P. W. Duin,et al.  Support Vector Data Description , 2004, Machine Learning.

[24]  Hongming Zhou,et al.  Optimization method based extreme learning machine for classification , 2010, Neurocomputing.

[25]  David B. Shmoys,et al.  A Best Possible Heuristic for the k-Center Problem , 1985, Math. Oper. Res..

[26]  Klaus-Robert Müller,et al.  A consistency-based model selection for one-class classification , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[27]  Danwei Wang,et al.  Sparse Extreme Learning Machine for Classification , 2014, IEEE Transactions on Cybernetics.

[28]  Malik Yousef,et al.  One-class document classification via Neural Networks , 2007, Neurocomputing.

[29]  Ye Yuan,et al.  An OS-ELM based distributed ensemble classification framework in P2P networks , 2011, Neurocomputing.

[30]  Ali Selamat,et al.  A hybrid model through the fusion of type-2 fuzzy logic systems and extreme learning machines for modelling permeability prediction , 2014, Inf. Fusion.

[31]  Shengjin Wang,et al.  Dynamic Extreme Learning Machine: A Learning Algorithm for Neural Network with Elastic Output Structure , 2009 .

[32]  Narasimhan Sundararajan,et al.  Online Sequential Fuzzy Extreme Learning Machine for Function Approximation and Classification Problems , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[33]  Robert P. W. Duin,et al.  Data description in subspaces , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[34]  Gang Yin,et al.  Online fault diagnosis method based on Incremental Support Vector Data Description and Extreme Learning Machine with incremental output structure , 2014, Neurocomputing.

[35]  Longbing Cao,et al.  T2FELA: Type-2 Fuzzy Extreme Learning Algorithm for Fast Training of Interval Type-2 TSK Fuzzy Logic System , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Shian-Shyong Tseng,et al.  Two-phase clustering process for outliers detection , 2001, Pattern Recognit. Lett..

[37]  Colin Campbell,et al.  A Linear Programming Approach to Novelty Detection , 2000, NIPS.

[38]  Narasimhan Sundararajan,et al.  Fully complex extreme learning machine , 2005, Neurocomputing.

[39]  Robert P. W. Duin,et al.  On the Choice of Smoothing Parameters for Parzen Estimators of Probability Density Functions , 1976, IEEE Transactions on Computers.

[40]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[41]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[42]  Narasimhan Sundararajan,et al.  An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[43]  Chandan Srivastava,et al.  Support Vector Data Description , 2011 .

[44]  Alexander Dekhtyar,et al.  Information Retrieval , 2018, Lecture Notes in Computer Science.

[45]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[46]  Peter L. Bartlett,et al.  The Sample Complexity of Pattern Classification with Neural Networks: The Size of the Weights is More Important than the Size of the Network , 1998, IEEE Trans. Inf. Theory.

[47]  Hyun Joon Shin,et al.  One-class support vector machines - an application in machine fault detection and classification , 2005, Comput. Ind. Eng..

[48]  Piotr Juszczak Learning to recognise : a study on one-class classification and active learning , 2006 .

[49]  Guang-Bin Huang,et al.  An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels , 2014, Cognitive Computation.

[50]  Gopinath Ganapathy,et al.  An efficient approach to an automatic detection of erythemato-squamous diseases , 2013, Neural Computing and Applications.

[51]  Erkki Oja,et al.  GPU-accelerated and parallelized ELM ensembles for large-scale regression , 2011, Neurocomputing.

[52]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[53]  E. Parzen On Estimation of a Probability Density Function and Mode , 1962 .

[54]  T. M. Williams,et al.  Practical Methods of Optimization. Vol. 1: Unconstrained Optimization , 1980 .

[55]  Robert P. W. Duin,et al.  One-Class LP Classifiers for Dissimilarity Representations , 2002, NIPS.