Multi-Label Classification Using Dependent and Independent Dual Space Reduction

While multi-label classification can be widely applied for problems where multiple classes can be assigned to an object, its effectiveness may be sacrificed due to curse of dimensionality in the feature space and sparseness of dimensionality in the label space. As a solution, this paper presents two alternative methods, namely Dependent Dual Space Reduction and Independent Dual Space Reduction, to reduce dimensions in the dual spaces, i.e., the feature and label spaces, using Singular Value Decomposition (SVD). The first approach constructs the cross-covariance matrix to represent dependency between the features and labels, projects both of them into a single reduced space, and then performs prediction on the reduced space. On the other hand, the second approach handles the feature space and the label space separately by constructing a covariance matrix for each space to represent feature dependency and label dependency before performing SVD on dependency profile of each space to reduce dimension and for noise elimination and then predicting classes using their reduced dimensions. A number of experiments show that prediction on the reduced spaces for both dependent and independent reduction approaches can obtain better classification accuracy as well as faster computation, compared to the prediction using the original spaces. Finally, the results of our methods are also compared to the prediction using other types of dimensionality reduction, with consideration of the factors of covariance and threshold selection.