Feature-Induced Labeling Information Enrichment for Multi-Label Learning

In multi-label learning, each training example is represented by a single instance (feature vector) while associated with multiple class labels simultaneously. The task is to learn a predictive model from the training examples which can assign a set of proper labels for the unseen instance. Most existing approaches make use of multi-label training examples by exploiting their labeling information in a crisp manner, i.e. one class label is either fully relevant or irrelevant to the instance. In this paper, a novel multi-label learning approach is proposed which aims to enrich the labeling information by leveraging the structural information in feature space. Firstly, the underlying structure of feature space is characterized by conducting sparse reconstruction among the training examples. Secondly, the reconstruction information is conveyed from feature space to label space so as to enrich the original categorical labels into numerical ones. Thirdly, the multilabel predictive model is induced by learning from training examples with enriched labeling information. Extensive experiments on fifteen benchmark data sets clearly validate the effectiveness of the proposed feature-induced strategy for enhancing labeling information of multi-label examples.

[1]  Sebastián Ventura,et al.  A Tutorial on Multilabel Learning , 2015, ACM Comput. Surv..

[2]  Fernando Pérez-Cruz,et al.  SVM multiregression for nonlinear channel estimation in multiple-input multiple-output systems , 2004, IEEE Transactions on Signal Processing.

[3]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[4]  Zhi-Hua Zhou,et al.  Multi-label Learning , 2017, Encyclopedia of Machine Learning and Data Mining.

[5]  Siegfried Gottwald,et al.  Fuzzy Sets and Fuzzy Logic , 1993 .

[6]  Miao Xu,et al.  Multi-Label Learning with PRO Loss , 2013, AAAI.

[7]  Xin Geng,et al.  Multi-Label Manifold Learning , 2016, AAAI.

[8]  Grigorios Tsoumakas,et al.  Random K-labelsets for Multilabel Classification , 2022 .

[9]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[10]  Lei Wu,et al.  Lift: Multi-Label Learning with Label-Specific Features , 2015, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Xin Geng,et al.  Label Distribution Learning , 2013, 2013 IEEE 13th International Conference on Data Mining Workshops.

[12]  Luis Alonso,et al.  Multioutput Support Vector Regression for Remote Sensing Biophysical Parameter Estimation , 2011, IEEE Geoscience and Remote Sensing Letters.

[13]  Euhanna Ghadimi,et al.  Optimal Parameter Selection for the Alternating Direction Method of Multipliers (ADMM): Quadratic Problems , 2013, IEEE Transactions on Automatic Control.

[14]  Zhang Yi,et al.  Fuzzy SVM with a new fuzzy membership function , 2006, Neural Computing & Applications.

[15]  Min-Ling Zhang,et al.  Leveraging Implicit Relative Labeling-Importance Information for Effective Multi-Label Learning , 2019 .

[16]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[17]  Euntai Kim,et al.  General Dimensional Multiple-Output Support Vector Regressions and Their Multiple Kernel Learning , 2015, IEEE Transactions on Cybernetics.

[18]  Jason Weston,et al.  A kernel method for multi-labelled classification , 2001, NIPS.

[19]  Eyke Hüllermeier,et al.  Multilabel classification via calibrated label ranking , 2008, Machine Learning.

[20]  George J. Klir,et al.  Fuzzy sets and fuzzy logic , 1995 .

[21]  Jieping Ye,et al.  A shared-subspace learning framework for multi-label classification , 2010, TKDD.

[22]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[23]  Eyke Hüllermeier,et al.  Graded Multilabel Classification: The Ordinal Case , 2010, ICML.

[24]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[25]  Geoff Holmes,et al.  Classifier chains for multi-label classification , 2009, Machine Learning.