Multilabel Feature Extraction Algorithm via Maximizing Approximated and Symmetrized Normalized Cross-Covariance Operator

Multilabel feature extraction (FE) is an effective preprocessing step to cope with some possible irrelevant, redundant, and noisy features, to reduce computational costs and even improve classification performance. Original normalized cross-covariance operator represents a kernel-based nonlinear dependence measure between features and labels, whose empirical estimator is formulated as a trace operation including two inverse matrices of feature and label kernels with a regularization constant. Due to such a complicated expression, it is impossible to derive an eigenvalue problem for linear FE directly. In this paper, we approximate this measure using Moore-Penrose inverse matrix, linear kernel for feature space, and delta kernel for label space, and then symmetrize the entire matrix in the trace operation, resulting in an effective approximated and symmetrized representation. According to orthonormal projection direction constraints, maximizing such a modified form induces a novel eigenvalue problem for multilabel linear FE. Experiments on 12 data sets illustrate that our proposed method works the best, compared with seven existing FE techniques, according to eight multilabel classification performance metrics and three statistical tests.