Discriminative concept factorization for data representation

Non-negative matrix factorization (NMF) has become a popular technique for finding low-dimensional representations of data. While the standard NMF can only be performed in the original feature space, one variant of NMF, named concept factorization, can be naturally kernelized and inherits all the strengths of NMF. To make use of label information, we propose a semi-supervised concept factorization technique called discriminative concept factorization (DCF) for data representation in this paper. DCF adopts a unified objective to combine the task of data reconstruction with the task of classification. These two tasks have mutual impacts on each other, which results in a concept factorization adapted to the classification task and a classifier built on the low-dimensional representations. Furthermore, we develop an iterative algorithm to solve the optimization problem through alternative convex programming. Experimental results on three real-word classification tasks demonstrate the effectiveness of DCF.

[1]  David G. Stork,et al.  Pattern Classification , 1973 .

[2]  Seungjin Choi,et al.  Semi-Supervised Nonnegative Matrix Factorization , 2010, IEEE Signal Processing Letters.

[3]  Jiawei Han,et al.  SRDA: An Efficient Algorithm for Large-Scale Discriminant Analysis , 2008, IEEE Transactions on Knowledge and Data Engineering.

[4]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[5]  Xuelong Li,et al.  Tensor Rank One Discriminant Analysis - A convergent method for discriminative multilinear subspace selection , 2008, Neurocomputing.

[6]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[7]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[8]  Jiawei Han,et al.  Locally Consistent Concept Factorization for Document Clustering , 2011, IEEE Transactions on Knowledge and Data Engineering.

[9]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[10]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[11]  Xuelong Li,et al.  Geometric Mean for Subspace Selection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Dong Xu,et al.  Semi-Supervised Bilinear Subspace Learning , 2009, IEEE Transactions on Image Processing.

[13]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[14]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[15]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[16]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[17]  Xuelong Li,et al.  L1-Norm-Based 2DPCA , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[18]  Shuicheng Yan,et al.  Non-Negative Semi-Supervised Learning , 2009, AISTATS.

[19]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[20]  Yihong Gong,et al.  Document clustering by concept factorization , 2004, SIGIR '04.

[21]  Guillermo Sapiro,et al.  Supervised Dictionary Learning , 2008, NIPS.

[22]  Chun Chen,et al.  Constrained Laplacian Eigenmap for dimensionality reduction , 2010, Neurocomputing.

[23]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Xuelong Li,et al.  Discriminant Locally Linear Embedding With High-Order Tensor Data , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Chun Chen,et al.  Graph Regularized Sparse Coding for Image Representation , 2011, IEEE Transactions on Image Processing.

[26]  Pablo Tamayo,et al.  Metagenes and molecular pattern discovery using matrix factorization , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[27]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[28]  I K Fodor,et al.  A Survey of Dimension Reduction Techniques , 2002 .

[29]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[30]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[31]  Daniel D. Lee,et al.  Multiplicative Updates for Nonnegative Quadratic Programming , 2007, Neural Computation.

[32]  Jiawei Han,et al.  Document clustering using locality preserving indexing , 2005, IEEE Transactions on Knowledge and Data Engineering.

[33]  Nathaniel E. Helwig,et al.  An Introduction to Linear Algebra , 2006 .

[34]  Harry Shum,et al.  Concurrent subspaces analysis , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[35]  Chun Chen,et al.  Discriminative codeword selection for image representation , 2010, ACM Multimedia.