Clustering with Similarity Preserving

Graph-based clustering has shown promising performance in many tasks. A key step of graph-based approach is the similarity graph construction. In general, learning graph in kernel space can enhance clustering accuracy due to the incorporation of nonlinearity. However, most existing kernel-based graph learning mechanisms is not similarity-preserving, hence leads to sub-optimal performance. To overcome this drawback, we propose a more discriminative graph learning method which can preserve the pairwise similarities between samples in an adaptive manner for the first time. Specifically, we require the learned graph be close to a kernel matrix, which serves as a measure of similarity in raw data. Moreover, the structure is adaptively tuned so that the number of connected components of the graph is exactly equal to the number of clusters. Finally, our method unifies clustering and graph learning which can directly obtain cluster indicators from the graph itself without performing further clustering step. The effectiveness of this approach is examined on both single and multiple kernel learning scenarios in several datasets.

[1]  Feiping Nie,et al.  Clustering and projected clustering with adaptive neighbors , 2014, KDD.

[2]  Ethem Alpaydin,et al.  Multiple Kernel Learning Algorithms , 2011, J. Mach. Learn. Res..

[3]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[4]  Yunming Ye,et al.  A feature group weighting method for subspace clustering of high-dimensional data , 2012, Pattern Recognit..

[5]  Yunming Ye,et al.  TW-k-means: Automated two-level variable weighting clustering algorithm for multiview data , 2013, IEEE Transactions on Knowledge and Data Engineering.

[6]  Kevin Chen-Chuan Chang,et al.  A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[7]  Bo Yang,et al.  Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering , 2016, ICML.

[8]  Rong Wang,et al.  Fast spectral clustering learning with hierarchical bipartite graph for large-scale data , 2020, Pattern Recognit. Lett..

[9]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[10]  Zenglin Xu,et al.  Low-rank kernel learning for graph-based clustering , 2019, Knowl. Based Syst..

[11]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[12]  Zhao Kang,et al.  Integrate and Conquer , 2018, ACM Trans. Intell. Syst. Technol..

[13]  Zhao Kang,et al.  Kernel-driven similarity learning , 2017, Neurocomputing.

[14]  R. Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications. , 2013, IEEE transactions on pattern analysis and machine intelligence.

[15]  Zenglin Xu,et al.  Adaptive local structure learning for document co-clustering , 2018, Knowl. Based Syst..

[16]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[17]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[18]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[19]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[20]  Zi Huang,et al.  Discrete Nonnegative Spectral Clustering , 2017, IEEE Transactions on Knowledge and Data Engineering.

[21]  K. Fan On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations I. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Juan C. Alvarado-Pérez,et al.  On the Relationship Between Dimensionality Reduction and Spectral Clustering from a Kernel Viewpoint , 2016, DCAI.

[23]  Feiping Nie,et al.  A New Simplex Sparse Learning Model to Measure Data Similarity for Clustering , 2015, IJCAI.

[24]  Zenglin Xu,et al.  Self-weighted multi-view clustering with soft capped norm , 2018, Knowl. Based Syst..

[25]  Johan A. K. Suykens,et al.  Kernel Spectral Clustering and applications , 2015, ArXiv.

[26]  K. Fan On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations: II. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[27]  Zenglin Xu,et al.  Auto-weighted multi-view clustering via kernelized graph learning , 2019, Pattern Recognit..

[28]  Zenglin Xu,et al.  Semi-supervised deep embedded clustering , 2019, Neurocomputing.

[29]  B. Mohar THE LAPLACIAN SPECTRUM OF GRAPHS y , 1991 .

[30]  Shuicheng Yan,et al.  Learning With $\ell ^{1}$-Graph for Image Analysis , 2010, IEEE Transactions on Image Processing.

[31]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[32]  Liang Du,et al.  Unsupervised Feature Selection with Adaptive Structure Learning , 2015, KDD.

[33]  Zenglin Xu,et al.  Self-weighted Multiple Kernel Learning for Graph-based Clustering and Semi-supervised Classification , 2018, IJCAI.

[34]  Zenglin Xu,et al.  Robust Graph Learning From Noisy Data , 2018, IEEE Transactions on Cybernetics.

[35]  Zenglin Xu,et al.  Unified Spectral Clustering with Optimal Graph , 2017, AAAI.

[36]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[37]  Zhao Kang,et al.  Twin Learning for Similarity and Clustering: A Unified Kernel Approach , 2017, AAAI.

[38]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[39]  Yung-Yu Chuang,et al.  Multiple Kernel Fuzzy Clustering , 2012, IEEE Transactions on Fuzzy Systems.

[40]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Lei Zhang,et al.  Sparse representation or collaborative representation: Which helps face recognition? , 2011, 2011 International Conference on Computer Vision.

[42]  Jane You,et al.  Adaptive Manifold Regularized Matrix Factorization for Data Clustering , 2017, IJCAI.

[43]  Gunnar Rätsch,et al.  Large Scale Multiple Kernel Learning , 2006, J. Mach. Learn. Res..

[44]  Zenglin Xu,et al.  Similarity Learning via Kernel Preserving Embedding , 2019, AAAI.

[45]  Jiangping Wang,et al.  Data Clustering by Laplacian Regularized L1-Graph , 2014, AAAI.

[46]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[47]  René Vidal,et al.  Structured Low-Rank Matrix Factorization: Global Optimality, Algorithms, and Applications , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Xinwang Liu,et al.  Learning a Joint Affinity Graph for Multiview Subspace Clustering , 2019, IEEE Transactions on Multimedia.

[49]  Lei Wang,et al.  On Similarity Preserving Feature Selection , 2013, IEEE Transactions on Knowledge and Data Engineering.

[50]  Johan A. K. Suykens,et al.  Multiway Spectral Clustering with Out-of-Sample Extensions through Weighted Kernel PCA , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Lei Shi,et al.  Robust Multiple Kernel K-means Using L21-Norm , 2015, IJCAI.

[52]  Yung-Yu Chuang,et al.  Affinity aggregation for spectral clustering , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[53]  Zihan Zhou,et al.  Label Information Guided Graph Construction for Semi-Supervised Learning , 2017, IEEE Transactions on Image Processing.

[54]  Xuelong Li,et al.  Multi-View Clustering and Semi-Supervised Classification with Adaptive Neighbours , 2017, AAAI.