Spectral clustering with discriminant cuts

Recently, many k-way spectral clustering algorithms have been proposed, satisfying one or both of the following requirements: between-cluster similarities are minimized and within-cluster similarities are maximized. In this paper, a novel graph-based spectral clustering algorithm called discriminant cut (Dcut) is proposed, which first builds the affinity matrix of a weighted graph and normalizes it with the corresponding regularized Laplacian matrix, then partitions the vertices into k parts. Dcut has several advantages. First, it is derived from graph partition and has a straightforward geometrical explanation. Second, it emphasizes the above requirements simultaneously. Besides, it is computationally feasible because the NP-hard intractable graph cut problem can be relaxed into a mild eigenvalue decomposition problem. Toy-data and real-data experimental results show that Dcut is pronounced comparing with other spectral clustering methods.

[1]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[2]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Allen S. Mandel Comment … , 1978, British heart journal.

[4]  Ying-Ke Lei,et al.  Semi-supervised locally discriminant projection for classification and recognition , 2011, Knowl. Based Syst..

[5]  Jianbo Shi,et al.  Multiclass spectral clustering , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[6]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[7]  Santosh S. Vempala,et al.  On clusterings: Good, bad and spectral , 2004, JACM.

[8]  Marina MeWi Comparing Clusterings , 2002 .

[9]  Wei Tang,et al.  Clusterer ensemble , 2006, Knowl. Based Syst..

[10]  Anna R. Karlin,et al.  Spectral analysis of data , 2001, STOC '01.

[11]  George L. Nemhauser,et al.  Min-cut clustering , 1993, Math. Program..

[12]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Chris H. Q. Ding,et al.  Spectral Relaxation for K-means Clustering , 2001, NIPS.

[14]  Aidong Zhang,et al.  Cluster analysis for gene expression data: a survey , 2004, IEEE Transactions on Knowledge and Data Engineering.

[15]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[16]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[17]  Jitendra Malik,et al.  Spectral grouping using the Nystrom method , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[19]  Yixin Chen,et al.  CLUE: cluster-based retrieval of images by unsupervised learning , 2005, IEEE Transactions on Image Processing.

[20]  Chris H. Q. Ding,et al.  A min-max cut algorithm for graph partitioning and data clustering , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[21]  C. Ding,et al.  Spectral relaxation models and structure analysis for K-way graph clustering and bi-clustering , 2001 .

[22]  Witold Pedrycz,et al.  Fuzzy Clustering With Viewpoints , 2010, IEEE Transactions on Fuzzy Systems.

[23]  David G. Stork,et al.  Pattern Classification , 1973 .

[24]  Andrew B. Kahng,et al.  New spectral methods for ratio cut partitioning and clustering , 1991, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[25]  Jun Guo,et al.  Learning a locality discriminating projection for classification , 2009, Knowl. Based Syst..

[26]  Qingsheng Zhu,et al.  Spectral clustering with density sensitive similarity function , 2011, Knowl. Based Syst..

[27]  B. Mohar,et al.  Some Applications of Laplace Eigenvalues of Graphs Some Applications of Laplace Eigenvalues of Graphs , 1997 .

[28]  Inderjit S. Dhillon,et al.  Kernel k-means: spectral clustering and normalized cuts , 2004, KDD.

[29]  Gene H. Golub,et al.  Matrix computations , 1983 .

[30]  David L. Wallace,et al.  A Method for Comparing Two Hierarchical Clusterings: Comment , 1983 .

[31]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[32]  Francesco Masulli,et al.  A survey of kernel and spectral methods for clustering , 2008, Pattern Recognit..

[33]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[34]  Multiway Spectral Clustering: A Margin-based Perspective , 2008, 1102.3768.

[35]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[36]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[37]  Santosh S. Vempala,et al.  On clusterings-good, bad and spectral , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[38]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[39]  Mikhail Belkin,et al.  Consistency of spectral clustering , 2008, 0804.0678.

[40]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[41]  Dan Klein,et al.  Spectral Learning , 2003, IJCAI.

[42]  C. Ding,et al.  A MinMaxCut Spectral Method for Data Clustering and Graph Partitioning , 2003 .

[43]  Jianbo Shi,et al.  A Random Walks View of Spectral Segmentation , 2001, AISTATS.

[44]  Michael I. Jordan,et al.  Learning Spectral Clustering, With Application To Speech Separation , 2006, J. Mach. Learn. Res..

[45]  Bernhard Schölkopf,et al.  Introduction to Semi-Supervised Learning , 2006, Semi-Supervised Learning.

[46]  Yair Weiss,et al.  Segmentation using eigenvectors: a unifying view , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[47]  Inderjit S. Dhillon,et al.  Weighted Graph Cuts without Eigenvectors A Multilevel Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Davi Geiger,et al.  Segmentation by grouping junctions , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[49]  Ping-Feng Pai,et al.  A support vector machine-based model for detecting top management fraud , 2011, Knowl. Based Syst..