Simultaneous Spectral Data Embedding and Clustering

Spectral clustering is often carried out by combining spectral data embedding and <inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula>-means clustering. However, the aims, dimensionality reduction and clustering, are usually not performed jointly. In this brief, we propose a novel approach to finding an optimal spectral embedding for identifying a partition of the set of objects; it iteratively alternates spectral embedding and clustering. In doing so, we show that our model can learn a low-dimensional representation more suited to clustering. Compared with classical spectral clustering methods, the proposed algorithm is not costly and outperforms not only these methods but also other <italic>nonnegative matrix factorization</italic> variants.

[1]  Chris H. Q. Ding,et al.  Nonnegative Matrix Factorization for Combinatorial Optimization: Spectral Clustering, Graph Matching, and Clique Finding , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[2]  Chris H. Q. Ding,et al.  A min-max cut algorithm for graph partitioning and data clustering , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[3]  Wei-Chien Chang.  On using Principal Components before Separating a Mixture of Two Multivariate Normal Distributions , 1983 .

[4]  Mohamed Nadif,et al.  Co-clustering under Nonnegative Matrix Tri-Factorization , 2011, ICONIP.

[5]  Mohamed Nadif,et al.  A Semi-NMF-PCA Unified Framework for Data Clustering , 2017, IEEE Transactions on Knowledge and Data Engineering.

[6]  Joydeep Ghosh,et al.  Cluster Ensembles --- A Knowledge Reuse Framework for Combining Multiple Partitions , 2002, J. Mach. Learn. Res..

[7]  Chris H. Q. Ding,et al.  Symmetric Nonnegative Matrix Factorization for Graph Clustering , 2012, SDM.

[8]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[9]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[10]  Erkki Oja,et al.  Projective Nonnegative Matrix Factorization for Image Compression and Feature Extraction , 2005, SCIA.

[11]  Michael I. Jordan,et al.  Learning Spectral Clustering, With Application To Speech Separation , 2006, J. Mach. Learn. Res..

[12]  Feiping Nie,et al.  On the eigenvectors of p-Laplacian , 2010, Machine Learning.

[13]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Mohamed Nadif,et al.  Power Simultaneous Spectral Data Embedding and Clustering , 2016, SDM.

[15]  Feiping Nie,et al.  Improved MinMax Cut Graph Clustering with Nonnegative Relaxation , 2010, ECML/PKDD.

[16]  C. Ding,et al.  On the Equivalence of Nonnegative Matrix Factorization and K-means - Spectral Clustering , 2005 .

[17]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Inderjit S. Dhillon,et al.  Co-clustering documents and words using bipartite spectral graph partitioning , 2001, KDD '01.

[19]  Mohamed Nadif,et al.  SemiNMF-PCA framework for Sparse Data Co-clustering , 2016, CIKM.

[20]  Martine D. F. Schlag,et al.  Spectral K-Way Ratio-Cut Partitioning and Clustering , 1993, 30th ACM/IEEE Design Automation Conference.

[21]  Jorma Laaksonen,et al.  Projective Non-Negative Matrix Factorization with Applications to Facial Image Processing , 2007, Int. J. Pattern Recognit. Artif. Intell..

[22]  Inderjit S. Dhillon,et al.  Kernel k-means: spectral clustering and normalized cuts , 2004, KDD.