Mutual-Information-Graph Regularized Sparse Transform for Unsupervised Feature Learning

Unsupervised feature learning is attracting more and more attention in machine learning and computer vision because of the increasing demand for effective representation of large-scale unlabeled data in real-world applications. This paper proposes a mutual-information-graph regularized sparse transform (MIST) algorithm by taking both of feature sparsity and underlying manifold structure of observation data into consideration. The feature transform is formulated by a transform kernel and a bias matrix. To obtain feature sparsity, the sparse filtering is utilized as nonlinear activation function. A mutual information graph is proposed to describe the underlying manifold structure of the observation data. The transform kernel and the bias matrix are finally learned under the regularization of the mutual information graph. The proposed approach has both the properties of sparsity and local-structure-preservation. These two properties guarantee the discriminative power and robustness in practical applications. Experimental results on handwritten digits recognition show that the proposed approach achieves high performance compared with existing unsupervised feature learning models.