Designing a Multi-label Kernel Machine with Two-Objective Optimization

In multi-label classification problems, some samples belong to multiple classes simultaneously and thus the classes are not mutually exclusive. How to characterize this kind of correlations between labels has been a key issue for designing a new multi-label classification approach. In this paper, we define two objective functions, i.e., the number of relevant and irrelevant label pairs which are ranked incorrectly, and the model regularization term, which depict the correlations between labels and the model complexity respectively. Then a new kernel machine for multi-label classification is constructed using two-objective minimization and solved by fast and elitist multi-objective genetic algorithm, i.e., NSGA-II. Experiments on the benchmark data set Yeast illustrate that our multi-label method is a competitive candidate for multilabel classification, compared with several state-of-the-art methods.

[1]  Zhi-Hua Zhou,et al.  A k-nearest neighbor based algorithm for multi-label classification , 2005, 2005 IEEE International Conference on Granular Computing.

[2]  Amanda Clare,et al.  Knowledge Discovery in Multi-label Phenotype Data , 2001, PKDD.

[3]  David W. Coit,et al.  Multi-objective optimization using genetic algorithms: A tutorial , 2006, Reliab. Eng. Syst. Saf..

[4]  S. Rahman.  Reliability Engineering and System Safety , 2011 .

[5]  Joshua D. Knowles,et al.  Multiobjective Optimization in Bioinformatics and Computational Biology , 2007, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[6]  Josef Kittler,et al.  Improving Multilabel Classification Performance by Using Ensemble of Multi-label Classifiers , 2010, MCS.

[7]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[8]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[9]  Zhi-Hua Zhou,et al.  Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization , 2006, IEEE Transactions on Knowledge and Data Engineering.

[10]  Jianhua Xu,et al.  A Fast Multi-label Classification Algorithm Based on Double Label Support Vector Machine , 2009, 2009 International Conference on Computational Intelligence and Security.

[11]  Eyke Hüllermeier,et al.  Combining Instance-Based Learning and Logistic Regression for Multilabel Classification , 2009, ECML/PKDD.

[12]  Jian-Hua Xu,et al.  A multi-label classification algorithm based on triple class support vector machine , 2007, 2007 International Conference on Wavelet Analysis and Pattern Recognition.

[13]  Yoram Singer,et al.  BoosTexter: A Boosting-based System for Text Categorization , 2000, Machine Learning.

[14]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[15]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[16]  Jan Komorowski,et al.  Principles of Data Mining and Knowledge Discovery , 2001, Lecture Notes in Computer Science.

[17]  Jason Weston,et al.  A kernel method for multi-labelled classification , 2001, NIPS.

[18]  Kalyanmoy Deb,et al.  Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms , 1994, Evolutionary Computation.

[19]  Rémi Gilleron,et al.  Learning Multi-label Alternating Decision Trees from Texts and Data , 2003, MLDM.

[20]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[21]  Víctor Robles,et al.  Feature selection for multi-label naive Bayes classification , 2009, Inf. Sci..