Joint Learning of Spectral Clustering Structure and Fuzzy Similarity Matrix of Data

When spectral clustering analysis is applied, a similarity matrix of data plays a vital role in both clustering performance and stability of clustering results. In order to enhance the clustering performance and maintain the stability of the clustering results, a new method to jointly learn the similarity matrix and the clustering structure, called the joint learning method (FSCM) of spectral clustering structure and fuzzy similarity matrix of data, is proposed in this paper. In FSCM, the capability of a double-index fuzzy C-means clustering algorithm is used to determine an appropriate fuzzy similarity between any pair of data points. A fuzzy similarity matrix of data is also determined by adaptively assigning fuzzy neighbors of data points so the spectral clustering structure of data can be found and the clustering stability of FSCM can be assured. Experimental results on synthetic and real datasets demonstrate the effectiveness of the proposed method.

[1]  L. Davis,et al.  Joint Image Clustering and Labeling by Matrix Factorization , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[3]  Zhaohong Deng,et al.  Knowledge-Leverage-Based Fuzzy System and Its Modeling , 2013, IEEE Transactions on Fuzzy Systems.

[4]  Zhaohong Deng,et al.  Double indices induced FCM clustering and its integration with fuzzy subspace clustering , 2012, FUZZ-IEEE.

[5]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[6]  Korris Fu-Lai Chung,et al.  On minimum class locality preserving variance support vector machine , 2010, Pattern Recognit..

[7]  K. Fan.  On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations I. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Chi-Hoon Lee,et al.  Clustering high dimensional data: A graph-based relaxed optimization approach , 2008, Inf. Sci..

[9]  Shanlin Yang,et al.  Fuzziness parameter selection in fuzzy c-means: The perspective of cluster validation , 2014, Science China Information Sciences.

[10]  Frank Klawonn,et al.  Improved fuzzy partitions for fuzzy regression models , 2003, Int. J. Approx. Reason..

[11]  Robert Jenssen,et al.  Mean shift spectral clustering , 2008, Pattern Recognit..

[12]  K. Fan.  On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations: II. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[13]  B. Mohar.  THE LAPLACIAN SPECTRUM OF GRAPHS y , 1991 .

[14]  W. T. Tucker,et al.  Convergence theory for fuzzy c-means: Counterexamples and repairs , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[15]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[16]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[17]  Chunyan Miao,et al.  Large Scale Document Categorization With Fuzzy Clustering , 2017, IEEE Transactions on Fuzzy Systems.

[18]  Larry S. Davis,et al.  Label Consistent K-SVD: Learning a Discriminative Dictionary for Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  S. C. Johnson.  Hierarchical clustering schemes , 1967, Psychometrika.

[20]  James M. Keller,et al.  A possibilistic approach to clustering , 1993, IEEE Trans. Fuzzy Syst..

[21]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Korris Fu-Lai Chung,et al.  Possibility Theoretic Clustering and its Preliminary Application to Large Image Segmentation , 2007, Soft Comput..

[23]  Andrew B. Kahng,et al.  New spectral methods for ratio cut partitioning and clustering , 1991, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[24]  Wei Hu,et al.  Unsupervised Active Learning Based on Hierarchical Graph-Theoretic Clustering , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Pengjiang Qian,et al.  Fast Graph-Based Relaxed Clustering for Large Data Sets Using Minimal Enclosing Ball , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  Zhaohong Deng,et al.  Robust maximum entropy clustering algorithm with its labeling for outliers , 2006, Soft Comput..

[27]  Dale Schuurmans,et al.  Maximum Margin Clustering , 2004, NIPS.

[28]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[29]  Joydeep Ghosh,et al.  Cluster Ensembles A Knowledge Reuse Framework for Combining Partitionings , 2002, AAAI/IAAI.

[30]  Marina Meila,et al.  l \ iultiway Cuts and Spectral Clustering , 2007 .

[31]  J. Douglas Carroll,et al.  Is the Distance Compression Effect Overstated? Some Theory and Experimentation , 2009, MLDM.

[32]  YU BIN,et al.  IMPACT OF REGULARIZATION ON SPECTRAL CLUSTERING , 2016 .

[33]  Shitong Wang,et al.  Attribute weighted mercer kernel based fuzzy clustering algorithm for general non-spherical datasets , 2006, Soft Comput..

[34]  Frank Chung-Hoon Rhee,et al.  Uncertain Fuzzy Clustering: Interval Type-2 Fuzzy Approach to $C$-Means , 2007, IEEE Transactions on Fuzzy Systems.

[35]  Zhaohong Deng,et al.  Semi-Supervised SVM With Extended Hidden Features , 2016, IEEE Transactions on Cybernetics.

[36]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[37]  Shanlin Yang,et al.  Exploring the uniform effect of FCM clustering: A data distribution perspective , 2016, Knowl. Based Syst..

[38]  Zhaohong Deng,et al.  Multitask TSK Fuzzy System Modeling by Mining Intertask Common Hidden Structure , 2015, IEEE Transactions on Cybernetics.

[39]  Hai Jin,et al.  Color Image Segmentation Based on Mean Shift and Normalized Cuts , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[40]  Weixin Xie,et al.  Suppressed fuzzy c-means clustering algorithm , 2003, Pattern Recognit. Lett..

[41]  Richard M. Leahy,et al.  An Optimal Graph Theoretic Approach to Data Clustering: Theory and Its Application to Image Segmentation , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Korris Fu-Lai Chung,et al.  Generalized Fuzzy C-Means Clustering Algorithm With Improved Fuzzy Partitions , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[43]  Punyaphol Horata,et al.  Harmonic extreme learning machine for data clustering , 2016, 2016 13th International Joint Conference on Computer Science and Software Engineering (JCSSE).

[44]  Korris Fu-Lai Chung,et al.  Generalized competitive agglomeration clustering algorithm , 2016, International Journal of Machine Learning and Cybernetics.

[45]  Takenobu Tokunaga,et al.  Hierarchical Bayesian Clustering for Automatic Text Classification , 1995, IJCAI.

[46]  Shifei Ding,et al.  Research on Spectral Clustering algorithms and prospects , 2010, 2010 2nd International Conference on Computer Engineering and Technology.

[47]  Yueting Zhuang,et al.  Data-Dependent Label Distribution Learning for Age Estimation , 2017, IEEE Transactions on Image Processing.

[48]  D. Higham,et al.  A Unified View of Spectral Clustering ∗ , 2004 .

[49]  Mohamed Ali Zoghlami,et al.  Sampling-based consensus fuzzy clustering on Big Data , 2016, 2016 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).