An Incremental Kernel Extreme Learning Machine for Multi-Label Learning With Emerging New Labels

Multi-label learning with emerging new labels is a practical problem that occurs in data streams and has become an important new research issue in the area of machine learning. However, existing models for dealing with this problem require high learning computational times, and there still exists a lack of research. Based on these issues, this paper presents an incremental kernel extreme learning machine for multi-label learning with emerging new labels, consisting of two parts: a novelty detector; and a multi-label classifier. The detector with free-user-setting threshold parameters was developed to identify instances with new labels. A new incremental multi-label classifier and its improved version were developed to predict a label set for each instance, which can add output units incrementally and update themselves in unlabeled instances. Comprehensive evaluations of the proposed method were carried out on the problems of multi-label classification with emerging new labels compared to comparative algorithms, which revealed the promising performance of the proposed method.

[1]  Saso Dzeroski,et al.  An extensive experimental comparison of methods for multi-label learning , 2012, Pattern Recognit..

[2]  Zhi-Hua Zhou,et al.  Classification Under Streaming Emerging New Classes: A Solution Using Completely-Random Trees , 2016, IEEE Transactions on Knowledge and Data Engineering.

[3]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[4]  Gang Yin,et al.  Online fault diagnosis method based on Incremental Support Vector Data Description and Extreme Learning Machine with incremental output structure , 2014, Neurocomputing.

[5]  Shifei Ding,et al.  Multi layer ELM-RBF for multi-label learning , 2016, Appl. Soft Comput..

[6]  Yanika Kongsorot,et al.  Kernel extreme learning machine based on fuzzy set theory for multi-label classification , 2019, Int. J. Mach. Learn. Cybern..

[7]  Geoff Holmes,et al.  Classifier chains for multi-label classification , 2009, Machine Learning.

[8]  Zhi-Hua Zhou,et al.  Isolation Forest , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[9]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[10]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Min-Ling Zhang,et al.  Ml-rbf: RBF Neural Networks for Multi-Label Learning , 2009, Neural Processing Letters.

[12]  Jun Feng,et al.  Extreme Learning Machine for Multi-Label Classification , 2016, Entropy.

[13]  Zhi-Hua Zhou,et al.  Multi-Label Learning with Emerging New Labels , 2018, IEEE Transactions on Knowledge and Data Engineering.

[14]  Sunita Sarawagi,et al.  Discriminative Methods for Multi-labeled Classification , 2004, PAKDD.

[15]  Francesco Camastra,et al.  Machine Learning for Audio, Image and Video Analysis - Theory and Applications , 2007, Advanced Information and Knowledge Processing.

[16]  Guang-Bin Huang,et al.  Classification ability of single hidden layer feedforward neural networks , 2000, IEEE Trans. Neural Networks Learn. Syst..

[17]  Chandan Srivastava,et al.  Support Vector Data Description , 2011 .

[18]  Jason Weston,et al.  A kernel method for multi-labelled classification , 2001, NIPS.

[19]  Shengjin Wang,et al.  Dynamic Extreme Learning Machine: A Learning Algorithm for Neural Network with Elastic Output Structure , 2009 .

[20]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[21]  Sebastián Ventura,et al.  A Tutorial on Multilabel Learning , 2015, ACM Comput. Surv..

[22]  Jun Miao,et al.  One-Class Classification with Extreme Learning Machine , 2015 .

[23]  Aruna Tiwari,et al.  On the construction of extreme learning machine for online and offline one-class classification - An expanded toolbox , 2017, Neurocomputing.

[24]  Shuangquan Wang,et al.  A Class Incremental Extreme Learning Machine for Activity Recognition , 2014, Cognitive Computation.

[25]  Zhi-Hua Zhou,et al.  Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization , 2006, IEEE Transactions on Knowledge and Data Engineering.