Discrete Optimal Graph Clustering

Graph-based clustering is one of the major clustering methods. Most of it works in three separate steps: 1) similarity graph construction; 2) clustering label relaxing; and 3) label discretization with ${k}$ -means (KM). Such common practice has three disadvantages: 1) the predefined similarity graph is often fixed and may not be optimal for the subsequent clustering; 2) the relaxing process of cluster labels may cause significant information loss; and 3) label discretization may deviate from the real clustering result since KM is sensitive to the initialization of cluster centroids. To tackle these problems, in this paper, we propose an effective discrete optimal graph clustering framework. A structured similarity graph that is theoretically optimal for clustering performance is adaptively learned with a guidance of reasonable rank constraints. Besides, to avoid the information loss, we explicitly enforce a discrete transformation on the intermediate continuous label, which derives a tractable optimization problem with a discrete solution. Furthermore, to compensate for the unreliability of the learned labels and enhance the clustering accuracy, we design an adaptive robust module that learns the prediction function for the unseen data based on the learned discrete cluster labels. Finally, an iterative optimization strategy guaranteed with convergence is developed to directly solve the clustering results. Extensive experiments conducted on both real and synthetic datasets demonstrate the superiority of our proposed methods compared with several state-of-the-art clustering approaches.

[1]  Feiping Nie,et al.  Clustering and projected clustering with adaptive neighbors , 2014, KDD.

[2]  Xuelong Li,et al.  Graph Regularized Non-Negative Low-Rank Matrix Factorization for Image Clustering , 2017, IEEE Transactions on Cybernetics.

[3]  Heng Tao Shen,et al.  Unsupervised Deep Hashing with Similarity-Adaptive and Discrete Optimization , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Yang Yang,et al.  Multitask Spectral Clustering by Exploring Intertask Correlation , 2015, IEEE Transactions on Cybernetics.

[5]  K. Fan On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations: II. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[6]  B. Mohar THE LAPLACIAN SPECTRUM OF GRAPHS y , 1991 .

[7]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[8]  Ke Lu,et al.  Transfer Independently Together: A Generalized Framework for Domain Adaptation , 2019, IEEE Transactions on Cybernetics.

[9]  Sylvain Guilley,et al.  First Principal Components Analysis: A New Side Channel Distinguisher , 2010, ICISC.

[10]  Tao Li,et al.  The Relationships Among Various Nonnegative Matrix Factorization Methods for Clustering , 2006, Sixth International Conference on Data Mining (ICDM'06).

[11]  Yang Yang,et al.  A Fast Optimization Method for General Binary Code Learning , 2016, IEEE Transactions on Image Processing.

[12]  Jieping Ye,et al.  Discriminative K-means for Clustering , 2007, NIPS.

[13]  Aidong Zhang,et al.  Cluster analysis for gene expression data: a survey , 2004, IEEE Transactions on Knowledge and Data Engineering.

[14]  Xuelong Li,et al.  Learning Discriminative Binary Codes for Large-scale Cross-modal Retrieval , 2017, IEEE Transactions on Image Processing.

[15]  Xiushan Nie,et al.  Efficient weakly-supervised discrete hashing for large-scale social image retrieval , 2020, Pattern Recognit. Lett..

[16]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[17]  Xuelong Li,et al.  A generalized power iteration method for solving quadratic problem on the Stiefel manifold , 2017, Science China Information Sciences.

[18]  Shuicheng Yan,et al.  Revisiting Jump-Diffusion Process for Visual Tracking: A Reinforcement Learning Approach , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Ivor W. Tsang,et al.  Spectral Embedded Clustering: A Framework for In-Sample and Out-of-Sample Spectral Clustering , 2011, IEEE Transactions on Neural Networks.

[20]  Lei Zhu,et al.  Task-Dependent and Query-Dependent Subspace Learning for Cross-Modal Retrieval , 2018, IEEE Access.

[21]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[22]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Yue Gao,et al.  Exploiting Web Images for Semantic Video Indexing Via Robust Sample-Specific Loss , 2014, IEEE Transactions on Multimedia.

[24]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[25]  Lei Zhu,et al.  Efficient discrete latent semantic hashing for scalable cross-modal retrieval , 2019, Signal Process..

[26]  Ke Lu,et al.  Heterogeneous Domain Adaptation Through Progressive Alignment , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Xuelong Li,et al.  Auto-Weighted Multi-View Learning for Image Clustering and Semi-Supervised Classification , 2018, IEEE Transactions on Image Processing.

[28]  Xinlei Chen,et al.  Large Scale Spectral Clustering Via Landmark-Based Sparse Representation , 2015, IEEE Transactions on Cybernetics.

[29]  Feiping Nie,et al.  The Constrained Laplacian Rank Algorithm for Graph-Based Clustering , 2016, AAAI.

[30]  Feiping Nie,et al.  Discriminative Unsupervised Dimensionality Reduction , 2015, IJCAI.

[31]  Feiping Nie,et al.  Spectral Rotation versus K-Means in Spectral Clustering , 2013, AAAI.

[32]  Jialie Shen,et al.  On Effective Location-Aware Music Recommendation , 2016, ACM Trans. Inf. Syst..

[33]  Bernhard Schölkopf,et al.  A Local Learning Approach for Clustering , 2006, NIPS.

[34]  Tao Mei,et al.  Just-for-Me: An Adaptive Personalization System for Location-Aware Social Music Recommendation , 2014, ICMR.

[35]  Andrew B. Kahng,et al.  New spectral methods for ratio cut partitioning and clustering , 1991, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[36]  Zenglin Xu,et al.  Unified Spectral Clustering with Optimal Graph , 2017, AAAI.

[37]  Wei Liu,et al.  Asymmetric Binary Coding for Image Search , 2017, IEEE Transactions on Multimedia.

[38]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[39]  Ke Lu,et al.  Low-Rank Discriminant Embedding for Multiview Learning , 2017, IEEE Transactions on Cybernetics.

[40]  Xuelong Li,et al.  Robust Web Image Annotation via Exploring Multi-Facet and Structural Knowledge , 2017, IEEE Transactions on Image Processing.

[41]  Wotao Yin,et al.  A feasible method for optimization with orthogonality constraints , 2013, Math. Program..

[42]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[43]  Yadong Mu,et al.  A Stochastic Attribute Grammar for Robust Cross-View Human Tracking , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[44]  K. Fan On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations I. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[45]  Zi Huang,et al.  Discrete Nonnegative Spectral Clustering , 2017, IEEE Transactions on Knowledge and Data Engineering.

[46]  Wei-Ying Ma,et al.  Annotating Images by Mining Image Search Results , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Fei Wang,et al.  Clustering with Local and Global Regularization , 2007, IEEE Transactions on Knowledge and Data Engineering.

[48]  Michael William Newman,et al.  The Laplacian spectrum of graphs , 2001 .