Robust Similarity-Based Concept Factorization for Data Representation

Non-negative matrix factorization (NMF) known as learnt parts-based representation has become a data analysis tool for clustering tasks. It provides an alternative learning paradigm to cope with non-negative data clustering. In this paradigm, concept factorization (CF) and symmetric non-negative factorization (SymNMF) are two typically important representative models. In general, they have distinct behaviors: in CF, each cluster is modeled as a linear combination of samples, and vice versa, i.e., sample reconstruction, while SymNMF built on pair-wise sample similarity measure, is to preserve similarity of samples in a low-dimensional subspace, namely similarity reconstruction. In this paper, we propose a similarity-based concept factorization (SCF) as a synthesis of the two behaviors. This design can be formulated as: the similarity of reconstructed samples by CF is close to that of original samples. To optimize it, we develop an optimization algorithm which leverages the alternating direction of multipliers (ADMM) method to solve each sub-problem of SCF. Besides, we take a further step to consider the robust issue of similarity reconstruction and explore a robust SCF model (RSCF), which penalizes the hardest pair-wise similarity reconstruction via $l_\infty $ . Thus, RSCF enjoys similarity preservation, robustness to similarity perturbation, and ability of reconstructing samples. Extensive experiments validate such properties and show that the proposed SCF and RSCF achieve large performance gains as compared to their counterparts.