Structured learning for unsupervised feature selection with high-order matrix factorization

Abstract Feature selection aims at searching the most discriminative and relevant features from high-dimensional data to improve the performance of certain learning tasks. Whereas, irrelevant or redundant features may increase the over-fitting risk of consequent learning algorithms. Structured learning of feature selection is to embed intrinsic structures of data, such as geometric structures and manifold structures, resulting in the improvement of learning performance. In this paper, three types of structured regularizers are embedded into the feature selection framework and an iterative algorithm with proved convergence for feature selection problem is proposed. First, serving as crucial representation pipelines of local structures, three types of local learning regularizers, including graph Laplacian, neighborhood preservation and sparsity regularizer, are defined. Second, the local and global structures are integrated into one joint framework for the feature selection problem. Third, the framework is formulated as the canonical form of high-order matrix factorizations and then an efficient convergent iterative algorithm is proposed for the problem. Besides, the proposed framework is further extended to multi-view feature selection and fusion problems from an algorithmic view. Finally, the proposed algorithm is tested on eight publicly available datasets and compared to several state-of-the-art feature selection methods. Experimental results demonstrate the superiority of the proposed method against the compared algorithms in terms of clustering performance.