Multiple Kernel Clustering With Global and Local Structure Alignment

Multiple kernel clustering (MKC) based on global structure alignment (GSA) has unified many existing MKC algorithms, and shown outstanding clustering performance. However, we observe that most of existing GSA-based MKC algorithms only maximally align global structure of data with an ideal similarity matrix, while ignoring the local geometrical structure hidden in data, which is regarded to be important in improving the clustering performance. To address this issue, we propose a global and local structure alignment framework for MKC (GLSAMKC) which well considers both the alignment between the global structure and local structure of data with the same ideal similarity matrix. To illustrate the effectiveness of the proposed framework, we instantiate two specific GLSAMKC-based algorithms by exploiting the local structure with local linear embedding and locality preserving projection, respectively. A two-step alternate iterative and convergent optimization algorithm is developed to implement the resultant optimization problem. Extensive experimental results on five benchmark data sets demonstrate the superiority of proposed algorithms compared with the many state-of-the-art MKC algorithms, indicating the effectiveness of the proposed framework.

[1]  Johan A. K. Suykens,et al.  Optimized Data Fusion for Kernel k-Means Clustering , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Dongyan Zhao,et al.  An overview of kernel alignment and its applications , 2012, Artificial Intelligence Review.

[3]  Zhengming Ma,et al.  Local Coordinates Alignment With Global Preservation for Dimensionality Reduction , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[4]  Enhong Chen,et al.  Learning Deep Representations for Graph Clustering , 2014, AAAI.

[5]  Michael W. Berry,et al.  Document clustering using nonnegative matrix factorization , 2006, Inf. Process. Manag..

[6]  Chenping Hou,et al.  Partial multi-view spectral clustering , 2018, Neurocomputing.

[7]  Lei Wang,et al.  Global and Local Structure Preservation for Feature Selection , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Lei Wang,et al.  Multiple Kernel Clustering with Local Kernel Alignment Maximization , 2016, IJCAI.

[9]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[10]  Inderjit S. Dhillon,et al.  Weighted Graph Cuts without Eigenvectors A Multilevel Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Jianping Yin,et al.  Robust local tangent space alignment via iterative weighted PCA , 2011, Neurocomputing.

[12]  Yong Dou,et al.  Optimal Neighborhood Kernel Clustering with Multiple Kernels , 2017, AAAI.

[13]  Mehmet Gönen,et al.  Localized Data Fusion for Kernel k-Means Clustering with Application to Cancer Biology , 2014, NIPS.

[14]  Jane You,et al.  Adaptive Manifold Regularized Matrix Factorization for Data Clustering , 2017, IJCAI.

[15]  Andries Petrus Engelbrecht,et al.  An overview of clustering methods , 2007, Intell. Data Anal..

[16]  Jing-Yu Yang,et al.  Multiple kernel clustering based on centered kernel alignment , 2014, Pattern Recognit..

[17]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[18]  Hal Daumé,et al.  A Co-training Approach for Multi-view Spectral Clustering , 2011, ICML.

[19]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[20]  Lei Wang,et al.  Multiple Kernel k-Means Clustering with Matrix-Induced Regularization , 2016, AAAI.

[21]  Lei Du,et al.  Robust Multi-View Spectral Clustering via Low-Rank and Sparse Decomposition , 2014, AAAI.

[22]  Feiping Nie,et al.  Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence Multi-View K-Means Clustering on Big Data , 2022 .

[23]  Houkuan Huang,et al.  Learning by local kernel polarization , 2009, Neurocomputing.

[24]  Lei Shi,et al.  Robust Multiple Kernel K-means Using L21-Norm , 2015, IJCAI.

[25]  Lei Shi,et al.  Recovery of Corrupted Multiple Kernels for Clustering , 2015, IJCAI.

[26]  Aristidis Likas,et al.  The Global Kernel $k$-Means Algorithm for Clustering in Feature Space , 2009, IEEE Transactions on Neural Networks.

[27]  Yung-Yu Chuang,et al.  Multiple Kernel Fuzzy Clustering , 2012, IEEE Transactions on Fuzzy Systems.

[28]  Feiping Nie,et al.  Multi-View Unsupervised Feature Selection with Adaptive Similarity and View Weight , 2017, IEEE Transactions on Knowledge and Data Engineering.

[29]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[30]  Jianqing Fan,et al.  A Selective Overview of Variable Selection in High Dimensional Feature Space. , 2009, Statistica Sinica.

[31]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.