Robust Bi-Stochastic Graph Regularized Matrix Factorization for Data Clustering

Data clustering, which is to partition the given data into different groups, has attracted much attention. Recently various effective algorithms have been developed to tackle the task. Among these methods, non-negative matrix factorization (NMF) has been demonstrated to be a powerful tool. However, there are still some problems. First, the standard NMF is sensitive to noises and outliers. Although L2,1 norm based NMF improves the robustness, it is still affected easily by large noises. Second, for most graph regularized NMF, the performance highly depends on the initial similarity graph. Third, many graph-based NMF models perform the graph construction and matrix factorization in two separated steps. Thus the learned graph structure may not be optimal. To overcome the above drawbacks, we propose a robust bi-stochastic graph regularized matrix factorization (RBSMF) framework for data clustering. Specifically, we present a general loss function, which is more robust than the commonly used L 2 and L 1 functions. Besides, instead of keeping the graph fixed, we learn an adaptive similarity graph. Furthermore, the graph updating and matrix factorization are processed simultaneously, which can make the learned graph more appropriate for clustering. Extensive experiments have shown the proposed RBSMF outperforms other state-of-the-art methods.

[1]  M. Hestenes Multiplier and gradient methods , 1969 .

[2]  S. C. Johnson Hierarchical clustering schemes , 1967, Psychometrika.

[3]  MengChu Zhou,et al.  An Efficient Non-Negative Matrix-Factorization-Based Approach to Collaborative Filtering for Recommender Systems , 2014, IEEE Transactions on Industrial Informatics.

[4]  Feiping Nie,et al.  Clustering and projected clustering with adaptive neighbors , 2014, KDD.

[5]  Xuelong Li,et al.  Graph Regularized Non-Negative Low-Rank Matrix Factorization for Image Clustering , 2017, IEEE Transactions on Cybernetics.

[6]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[7]  Xingpeng Jiang,et al.  Document Clustering with an Augmented Nonnegative Matrix Factorization Model , 2014, PAKDD.

[8]  Hong Yu,et al.  Multi-view clustering via multi-manifold regularized non-negative matrix factorization , 2017, Neural Networks.

[9]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Zhenyue Zhang,et al.  Low-Rank Matrix Approximation with Manifold Regularization , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Chris H. Q. Ding,et al.  On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering , 2005, SDM.

[13]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[14]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[15]  Rongrong Ji,et al.  Robust nonnegative matrix factorization via L1 norm regularization by multiplicative updating rules , 2012, 2014 IEEE International Conference on Image Processing (ICIP).

[16]  Fei Wang,et al.  Improving clustering by learning a bi-stochastic data similarity matrix , 2011, Knowledge and Information Systems.

[17]  Zenglin Xu,et al.  Nonnegative matrix factorization with adaptive neighbors , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[18]  A. Ben Hamza,et al.  Reconstruction of reflectance spectra using robust nonnegative matrix factorization , 2006, IEEE Transactions on Signal Processing.

[19]  Feiping Nie,et al.  The Constrained Laplacian Rank Algorithm for Graph-Based Clustering , 2016, AAAI.

[20]  Zenglin Xu,et al.  Robust graph regularized nonnegative matrix factorization for clustering , 2017, Data Mining and Knowledge Discovery.

[21]  Xi Chen,et al.  Direct Robust Matrix Factorizatoin for Anomaly Detection , 2011, 2011 IEEE 11th International Conference on Data Mining.

[22]  Zhengyou Zhang,et al.  Parameter estimation techniques: a tutorial with application to conic fitting , 1997, Image Vis. Comput..

[23]  Hans-Peter Kriegel,et al.  Subspace clustering , 2012, WIREs Data Mining Knowl. Discov..

[24]  Fei Wang,et al.  Graph dual regularization non-negative matrix factorization for co-clustering , 2012, Pattern Recognit..

[25]  Xiaojun Wu,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[27]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[28]  D.M. Mount,et al.  An Efficient k-Means Clustering Algorithm: Analysis and Implementation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Xiaofei He,et al.  Robust non-negative matrix factorization , 2011 .

[30]  Amnon Shashua,et al.  Doubly Stochastic Normalization for Spectral Clustering , 2006, NIPS.

[31]  Badong Chen,et al.  Robust sparse nonnegative matrix factorization based on maximum correntropy criterion , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[32]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[33]  MengChu Zhou,et al.  Generating Highly Accurate Predictions for Missing QoS Data via Aggregating Nonnegative Latent Factor Models , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[35]  Feiping Nie,et al.  Robust Manifold Nonnegative Matrix Factorization , 2014, ACM Trans. Knowl. Discov. Data.

[36]  MengChu Zhou,et al.  A Nonnegative Latent Factor Model for Large-Scale Sparse Matrices in Recommender Systems via Alternating Direction Method , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[37]  Martine D. F. Schlag,et al.  Spectral K-Way Ratio-Cut Partitioning and Clustering , 1993, 30th ACM/IEEE Design Automation Conference.

[38]  J. Neumann Functional Operators (AM-22), Volume 2: The Geometry of Orthogonal Spaces. (AM-22) , 1951 .

[39]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[40]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[41]  Feiping Nie,et al.  Structured Doubly Stochastic Matrix for Graph Based Clustering: Structured Doubly Stochastic Matrix , 2016, KDD.

[42]  Xuelong Li,et al.  Large Sparse Cone Non-negative Matrix Factorization for Image Annotation , 2017, ACM Trans. Intell. Syst. Technol..

[43]  Xuelong Li,et al.  Anchor-based group detection in crowd scenes , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[44]  Chris H. Q. Ding,et al.  Robust nonnegative matrix factorization using L21-norm , 2011, CIKM '11.