Learning the sparse representation for classification

In this work, we propose a novel supervised matrix factorization method used directly as a multi-class classifier. The coefficient matrix of the factorization is enforced to be sparse by ℓ1-norm regularization. The basis matrix is composed of atom dictionaries from different classes, which are trained in a jointly supervised manner by penalizing inhomogeneous representations given the labeled data samples. The learned basis matrix models the data of interest as a union of discriminative linear subspaces by sparse projection. The proposed model is based on the observation that many high-dimensional natural signals lie in a much lower dimensional subspaces or union of subspaces. Experiments conducted on several datasets show the effectiveness of such a representation model for classification, which also suggests that a tight reconstructive representation model could be very useful for discriminant analysis.