Flexible Auto-Weighted Local-Coordinate Concept Factorization: A Robust Framework for Unsupervised Clustering

Concept Factorization (CF) and its variants may produce inaccurate representation and clustering results due to the sensitivity to noise, hard constraint on the reconstruction error and pre-obtained approximate similarities. To improve the representation ability, a novel unsupervised Robust Flexible Auto-weighted Local-coordinate Concept Factorization (RFA-LCF) framework is proposed for clustering high-dimensional data. Specifically, RFA-LCF integrates the robust flexible CF by clean data space recovery, robust sparse local-coordinate coding and adaptive weighting into a unified model. RFA-LCF improves the representations by enhancing the robustness of CF to noise and errors, providing a flexible constraint on the reconstruction error and optimizing the locality jointly. For robust learning, RFA-LCF clearly learns a sparse projection to recover the underlying clean data space, and then the flexible CF is performed in the projected feature space. RFA-LCF also uses a L2,1-norm based flexible residue to encode the mismatch between the recovered data and its reconstruction, and uses the robust sparse local-coordinate coding to represent data using a few nearby basis concepts. For auto-weighting, RFA-LCF jointly preserves the manifold structures in the basis concept space and new coordinate space in an adaptive manner by minimizing the reconstruction errors on clean data, anchor points and coordinates. By updating the local-coordinate preserving data, basis concepts and new coordinates alternately, the representation abilities can be potentially improved. Extensive results on public databases show that RFA-LCF delivers the state-of-the-art clustering results compared with other related methods.

[1]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[2]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[3]  Duc Truong Pham,et al.  Control chart pattern recognition using a new type of self-organizing neural network , 1998 .

[4]  Qingshan Liu,et al.  Robust Subspace Clustering With Compressed Data , 2018, IEEE Transactions on Image Processing.

[5]  Dejun Mu,et al.  Multi-document summarization based on sentence cluster using non-negative matrix factorization , 2017, J. Intell. Fuzzy Syst..

[6]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[7]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[8]  Zi Huang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ℓ2,1-Norm Regularized Discriminative Feature Selection for Unsupervised Learning , 2022 .

[9]  Masashi Sugiyama,et al.  Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis , 2007, J. Mach. Learn. Res..

[10]  Jun Ye,et al.  Graph-Regularized Local Coordinate Concept Factorization for Image Representation , 2017, Neural Processing Letters.

[11]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[12]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[13]  Hongbin Zha,et al.  Robust Matrix Factorization by Majorization Minimization , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Binbin Pan,et al.  Supervised kernel nonnegative matrix factorization for face recognition , 2016, Neurocomputing.

[15]  Feiping Nie,et al.  The Constrained Laplacian Rank Algorithm for Graph-Based Clustering , 2016, AAAI.

[16]  Huirong Li,et al.  Graph-regularized CF with local coordinate for image representation , 2017, J. Vis. Commun. Image Represent..

[17]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[18]  Erkki Oja,et al.  Linear and Nonlinear Projective Nonnegative Matrix Factorization , 2010, IEEE Transactions on Neural Networks.

[19]  Xuelong Li,et al.  Constrained Nonnegative Matrix Factorization for Image Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Bernt Schiele,et al.  Analyzing appearance and contour based methods for object categorization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[21]  Cong Hu,et al.  Structure Preserving Sparse Coding for Data Representation , 2018, Neural Processing Letters.

[22]  Xiaojun Wu,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Konstantinos N. Plataniotis,et al.  Face recognition using LDA-based algorithms , 2003, IEEE Trans. Neural Networks.

[24]  Zhao Zhang,et al.  A Sparse Projection and Low-Rank Recovery Framework for Handwriting Representation and Salient Stroke Feature Extraction , 2015, ACM Trans. Intell. Syst. Technol..

[25]  Li Zhang,et al.  Joint Label Consistent Dictionary Learning and Adaptive Label Prediction for Semisupervised Machine Fault Classification , 2016, IEEE Transactions on Industrial Informatics.

[26]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[27]  Gene H. Golub,et al.  Singular value decomposition and least squares solutions , 1970, Milestones in Matrix Computation.

[28]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[29]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[30]  Zhao Zhang,et al.  Joint Subspace Recovery and Enhanced Locality Driven Robust Flexible Discriminative Dictionary Learning , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Ivor W. Tsang,et al.  Flexible Manifold Embedding: A Framework for Semi-Supervised and Unsupervised Dimension Reduction , 2010, IEEE Transactions on Image Processing.

[32]  Shuicheng Yan,et al.  Jointly Learning Structured Analysis Discriminative Dictionary and Analysis Multiclass Classifier , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[33]  Li Zhang,et al.  Joint Low-Rank and Sparse Principal Feature Coding for Enhanced Robust Representation and Visual Classification , 2016, IEEE Transactions on Image Processing.

[34]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Yihong Gong,et al.  Document clustering by concept factorization , 2004, SIGIR '04.

[36]  Hsien-Chung Wu,et al.  The Karush-Kuhn-Tucker optimality conditions in an optimization problem with interval-valued objective function , 2007, Eur. J. Oper. Res..

[37]  Xiaobing Pei,et al.  Concept Factorization With Adaptive Neighbors for Document Clustering , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[38]  Vicki Bruce,et al.  Face Recognition: From Theory to Applications , 1999 .

[39]  R. Gray,et al.  Vector quantization , 1984, IEEE ASSP Magazine.

[40]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[41]  Meng Wang,et al.  Robust Unsupervised Flexible Auto-weighted Local-coordinate Concept Factorization for Image Clustering , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[42]  Yan Zhang,et al.  Discriminative sparse flexible manifold embedding with novel graph for robust visual representation and label propagation , 2017, Pattern Recognit..

[43]  Fei Wang,et al.  Graph dual regularization non-negative matrix factorization for co-clustering , 2012, Pattern Recognit..

[44]  Larry S. Davis,et al.  Label Consistent K-SVD: Learning a Discriminative Dictionary for Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Jiawei Han,et al.  Locally Consistent Concept Factorization for Document Clustering , 2011, IEEE Transactions on Knowledge and Data Engineering.

[46]  Meng Wang,et al.  Joint Label Prediction Based Semi-Supervised Adaptive Concept Factorization for Robust Data Representation , 2019, IEEE Transactions on Knowledge and Data Engineering.

[47]  Haibo Wang,et al.  Adaptive Structure Concept Factorization for Multiview Clustering , 2018, Neural Computation.

[48]  Jiawei Han,et al.  Document clustering using locality preserving indexing , 2005, IEEE Transactions on Knowledge and Data Engineering.

[49]  Tinne Hoff Kjeldsen,et al.  Traces and Emergence of Nonlinear Programming , 2013 .

[50]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[51]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[52]  Binbin Pan,et al.  Fast Non-Negative Matrix Factorizations for Face Recognition , 2018, Int. J. Pattern Recognit. Artif. Intell..

[53]  Shuyuan Yang,et al.  Feature selection based dual-graph sparse non-negative matrix factorization for local discriminative clustering , 2018, Neurocomputing.

[54]  Jean-Yves Tourneret,et al.  Motion Estimation in Echocardiography Using Sparse Representation and Dictionary Learning , 2018, IEEE Transactions on Image Processing.

[55]  Jun Ye,et al.  Dual-graph regularized concept factorization for clustering , 2014, Neurocomputing.

[56]  Xuelong Li,et al.  Local Coordinate Concept Factorization for Image Representation , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[57]  Zhaohui Wu,et al.  Constrained Concept Factorization for Image Representation , 2014, IEEE Transactions on Cybernetics.

[58]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[59]  Zhiping Lin,et al.  An adaptive graph learning method based on dual data representations for clustering , 2018, Pattern Recognit..