Simultaneous Subspace Clustering and Cluster Number Estimating Based on Triplet Relationship

In this paper, we propose a unified framework to discover the number of clusters and group the data points into different clusters using subspace clustering simultaneously. Real data distributed in a high-dimensional space can be disentangled into a union of low-dimensional subspaces, which can benefit various applications. To explore such intrinsic structure, state-of-the-art subspace clustering approaches often optimize a self-representation problem among all samples, to construct a pairwise affinity graph for spectral clustering. However, a graph with pairwise similarities lacks robustness for segmentation, especially for samples which lie on the intersection of two subspaces. To address this problem, we design a hyper-correlation-based data structure termed as the triplet relationship, which reveals high relevance and local compactness among three samples. The triplet relationship can be derived from the self-representation matrix, and be utilized to iteratively assign the data points to clusters. Based on the triplet relationship, we propose a unified optimizing scheme to automatically calculate clustering assignments. Specifically, we optimize a model selection reward and a fusion reward by simultaneously maximizing the similarity of triplets from different clusters while minimizing the correlation of triplets from the same cluster. The proposed algorithm also automatically reveals the number of clusters and fuses groups to avoid over-segmentation. Extensive experimental results on both synthetic and real-world datasets validate the effectiveness and robustness of the proposed method.

[1]  A. A. Abin,et al.  Querying Beneficial Constraints Before Clustering Using Facility Location Analysis , 2018, IEEE Transactions on Cybernetics.

[2]  Junbin Gao,et al.  Product Grassmann Manifold Representation and Its LRR Models , 2016, AAAI.

[3]  Xiaochun Cao,et al.  Constrained Multi-View Video Face Clustering , 2015, IEEE Transactions on Image Processing.

[4]  Qinghua Hu,et al.  Subspace clustering guided unsupervised feature selection , 2017, Pattern Recognit..

[5]  Constantine Caramanis,et al.  Greedy Subspace Clustering , 2014, NIPS.

[6]  Ke Chen,et al.  Reweighted sparse subspace clustering , 2015, Comput. Vis. Image Underst..

[7]  Marimuthu Palaniswami,et al.  A Hybrid Approach to Clustering in Big Data , 2016, IEEE Transactions on Cybernetics.

[8]  Hans-Peter Kriegel,et al.  Subspace clustering , 2012, WIREs Data Mining Knowl. Discov..

[9]  Aswin C. Sankaranarayanan,et al.  Greedy feature selection for subspace clustering , 2013, J. Mach. Learn. Res..

[10]  Hongbin Zha,et al.  A Unified Convex Surrogate for the Schatten-p Norm , 2016, AAAI.

[11]  Soon Ki Jung,et al.  Background–Foreground Modeling Based on Spatiotemporal Sparse Subspace Clustering , 2017, IEEE Transactions on Image Processing.

[12]  Xuelong Li,et al.  Graph Regularized Non-Negative Low-Rank Matrix Factorization for Image Clustering , 2017, IEEE Transactions on Cybernetics.

[13]  Hongchuan Yu,et al.  Constrained Low-Rank Representation for Robust Subspace Clustering , 2017, IEEE Transactions on Cybernetics.

[14]  Tomoyuki Obuchi,et al.  Mean-field theory of graph neural networks in graph partitioning , 2018, NeurIPS.

[15]  Zhuwen Li,et al.  Simultaneous Clustering and Model Selection: Algorithm, Theory and Applications , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Kai Wang,et al.  Automatic Model Selection in Subspace Clustering via Triplet Relationships , 2018, AAAI.

[17]  René Vidal,et al.  Structured Sparse Subspace Clustering: A unified optimization framework , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Shuicheng Yan,et al.  Robust Subspace Segmentation with Block-Diagonal Prior , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Yang Yang,et al.  Multitask Spectral Clustering by Exploring Intertask Correlation , 2015, IEEE Transactions on Cybernetics.

[20]  Witold Pedrycz,et al.  A Clustering-Based Graph Laplacian Framework for Value Function Approximation in Reinforcement Learning , 2014, IEEE Transactions on Cybernetics.

[21]  Zhang Yi,et al.  Scalable Sparse Subspace Clustering , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[23]  Yong Tang,et al.  Efficient k-Support Matrix Pursuit , 2014, ECCV.

[24]  George Atia,et al.  Innovation Pursuit: A New Approach to the Subspace Clustering Problem , 2017, ICML.

[25]  Kai Wang,et al.  Sub-GAN: An Unsupervised Generative Model via Subspaces , 2018, ECCV.

[26]  Tao Li,et al.  Constraint Neighborhood Projections for Semi-Supervised Clustering , 2014, IEEE Transactions on Cybernetics.

[27]  Shuicheng Yan,et al.  Correlation Adaptive Subspace Segmentation by Trace Lasso , 2013, 2013 IEEE International Conference on Computer Vision.

[28]  Huan Xu,et al.  Provable Subspace Clustering: When LRR Meets SSC , 2013, IEEE Transactions on Information Theory.

[29]  René Vidal,et al.  A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Junwei Han,et al.  A Unified Metric Learning-Based Framework for Co-Saliency Detection , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Minsik Lee,et al.  Membership representation for detecting block-diagonal structure in low-rank or sparse subspace clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Richard I. Hartley,et al.  Graph connectivity in sparse subspace clustering , 2011, CVPR 2011.

[33]  Yin Wang,et al.  Subspace Clustering with Priors via Sparse Quadratically Constrained Quadratic Programming , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Sean Hughes,et al.  Clustering by Fast Search and Find of Density Peaks , 2016 .

[35]  Junbin Gao,et al.  Spatial subspace clustering for drill hole spectral data , 2014 .

[36]  Dong Xu,et al.  FaLRR: A fast low rank representation solver , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Junbin Gao,et al.  SPATIAL SUBSPACE CLUSTERING FOR HYPERSPECTRAL DATA SEGMENTATION , 2013 .

[38]  René Vidal,et al.  Structured Sparse Subspace Clustering: A Joint Affinity Learning and Subspace Clustering Framework , 2016, IEEE Transactions on Image Processing.

[39]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[40]  Sebastian Nowozin,et al.  Image Segmentation UsingHigher-Order Correlation Clustering , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Kun Zhan,et al.  Graph Learning for Multiview Clustering , 2018, IEEE Transactions on Cybernetics.

[43]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Liang-Tien Chia,et al.  Laplacian Sparse Coding, Hypergraph Laplacian Sparse Coding, and Applications , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Tat-Jun Chin,et al.  Clustering with Hypergraphs: The Case for Large Hyperedges , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Hong Jia,et al.  Subspace Clustering of Categorical and Numerical Data With an Unknown Number of Clusters , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[47]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[48]  Junwei Han,et al.  CNNs-Based RGB-D Saliency Detection via Cross-View Transfer and Multiview Fusion. , 2018, IEEE transactions on cybernetics.

[49]  Zhang Yi,et al.  Constructing the L2-Graph for Robust Subspace Learning and Subspace Clustering , 2012, IEEE Transactions on Cybernetics.

[50]  Wei Liu,et al.  Tensor Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Tensors via Convex Optimization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Songhwai Oh,et al.  Robust Elastic-Net Subspace Representation , 2016, IEEE Transactions on Image Processing.

[52]  Daniel P. Robinson,et al.  Oracle Based Active Set Algorithm for Scalable Elastic Net Subspace Clustering , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Qingshan Liu,et al.  A Deterministic Analysis for LRR , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Shuicheng Yan,et al.  Robust and Efficient Subspace Segmentation via Least Squares Regression , 2012, ECCV.

[55]  Honggang Zhang,et al.  Learning Bundle Manifold by Double Neighborhood Graphs , 2009, ACCV.

[56]  Nebojsa Jojic,et al.  -Sparse Subspace Clustering , 2016 .

[57]  Qingshan Liu,et al.  Decentralized Robust Subspace Clustering , 2016, AAAI.

[58]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[59]  Ming-Hsuan Yang,et al.  Dynamic Match Kernel With Deep Convolutional Features for Image Retrieval , 2018, IEEE Transactions on Image Processing.

[60]  Shuicheng Yan,et al.  Robust Clustering as Ensembles of Affinity Relations , 2010, NIPS.

[61]  Bernhard Schölkopf,et al.  Learning with Hypergraphs: Clustering, Classification, and Embedding , 2006, NIPS.

[62]  Stan Z. Li,et al.  Exclusivity-Consistency Regularized Multi-view Subspace Clustering , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[63]  René Vidal,et al.  A closed form solution to robust subspace estimation and clustering , 2011, CVPR 2011.

[64]  René Vidal,et al.  Sparse Manifold Clustering and Embedding , 2011, NIPS.

[65]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[66]  Junbin Gao,et al.  Ordered Subspace Clustering With Block-Diagonal Priors , 2016, IEEE Transactions on Cybernetics.

[67]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[68]  Paul M. B. Vitányi,et al.  Clustering by compression , 2003, IEEE Transactions on Information Theory.

[69]  Zhuwen Li,et al.  Simultaneous Clustering and Model Selection for Tensor Affinities , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[70]  Dong Xu,et al.  Learning Rotation-Invariant and Fisher Discriminative Convolutional Neural Networks for Object Detection , 2019, IEEE Transactions on Image Processing.

[71]  Zhuwen Li,et al.  SCAMS: Simultaneous Clustering and Model Selection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[72]  René Vidal,et al.  A Structured Sparse Plus Structured Low-Rank Framework for Subspace Clustering and Completion , 2016, IEEE Transactions on Signal Processing.

[73]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[74]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[75]  Deyu Meng,et al.  Leveraging Prior-Knowledge for Weakly Supervised Object Detection Under a Collaborative Self-Paced Curriculum Learning Framework , 2018, International Journal of Computer Vision.

[76]  Jianjiang Feng,et al.  Smooth Representation Clustering , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[77]  Daniel P. Robinson,et al.  Scalable Sparse Subspace Clustering by Orthogonal Matching Pursuit , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[78]  Xuelong Li,et al.  Robust Semi-Supervised Subspace Clustering via Non-Negative Low-Rank Representation , 2016, IEEE Transactions on Cybernetics.

[79]  Ehsan Elhamifar,et al.  Sparse subspace clustering , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[80]  Geoffrey E. Hinton,et al.  Zero-shot Learning with Semantic Output Codes , 2009, NIPS.

[81]  Jörg H. Kappes,et al.  Fusion moves for correlation clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[82]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[83]  Jun Zhu,et al.  DP-space: Bayesian Nonparametric Subspace Clustering with Small-variance Asymptotics , 2015, ICML.

[84]  Qinghua Hu,et al.  Flexible Multi-View Dimensionality Co-Reduction , 2017, IEEE Transactions on Image Processing.