Entropy-based active sparse subspace clustering

Sparse Subspace Clustering (SSC) is widely used in data mining and machine learning. Some studies have been developed to add pairwise constraints as side information to improve the clustering results. However, most of these algorithms are “passive” in the sense that the side information is provided beforehand. In this paper, we propose a novel extension for SSC with active learning framework, in which we aim to select the most informative pairwise constraints to guide the SSC for accurate clustering results. Specifically, in the first step, an entropy-based query strategy is proposed to select the most uncertain pairwise constraints. Next, constrained sparse subspace clustering algorithms are followed to integrate the selected pairwise constraints and obtain the final clustering results. Two steps are effectively performed in an iterative manner until satisfactory results are achieved. Experimental results on two face datasets clustering well demonstrate the effectiveness of the proposed method.

[1]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Dan Klein,et al.  From Instance-level Constraints to Space-Level Constraints: Making the Most of Prior Knowledge in Data Clustering , 2002, ICML.

[3]  Tao Li,et al.  Entropy-based criterion in categorical clustering , 2004, ICML.

[4]  Arindam Banerjee,et al.  Active Semi-Supervision for Pairwise Constrained Clustering , 2004, SDM.

[5]  Inderjit S. Dhillon,et al.  Semi-supervised graph clustering: a kernel approach , 2005, ICML '05.

[6]  Marie desJardins,et al.  Active Constrained Clustering by Examining Spectral Eigenvectors , 2005, Discovery Science.

[7]  D. Donoho For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[8]  Wai Lam,et al.  Semi-supervised Document Clustering via Active Learning with Pairwise Constraints , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[9]  Chris H. Q. Ding,et al.  Solving Consensus and Semi-supervised Clustering Problems Using Nonnegative Matrix Factorization , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[10]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression , 2007, J. Mach. Learn. Res..

[11]  Derek Greene,et al.  Constraint Selection by Committee: An Ensemble Approach to Identifying Informative Constraints for Semi-supervised Clustering , 2007, ECML.

[12]  Yuxiao Hu,et al.  Learning a Spatially Smooth Subspace for Face Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Nozha Boujemaa,et al.  Active semi-supervised fuzzy clustering , 2008, Pattern Recognit..

[14]  Rong Jin,et al.  Active kernel learning , 2008, ICML '08.

[15]  Guangliang Chen,et al.  Spectral Curvature Clustering (SCC) , 2009, International Journal of Computer Vision.

[16]  Rong Jin,et al.  Active query selection for semi-supervised clustering , 2008, 2008 19th International Conference on Pattern Recognition.

[17]  René Vidal,et al.  Sparse subspace clustering , 2009, CVPR.

[18]  Yi Yang,et al.  Image Clustering Using Local Discriminant Models and Global Integration , 2010, IEEE Transactions on Image Processing.

[19]  Ian Davidson,et al.  Active Spectral Clustering , 2010, 2010 IEEE International Conference on Data Mining.

[20]  Robert D. Nowak,et al.  Active Clustering: Robust and Efficient Hierarchical Clustering using Adaptively Selected Similarities , 2011, AISTATS.

[21]  Young-Rae Cho,et al.  Entropy-Based Graph Clustering: Application to Biological and Social Networks , 2011, 2011 IEEE 11th International Conference on Data Mining.

[22]  Large Scale Image Clustering with Active Pairwise Constraints , 2011 .

[23]  David W. Jacobs,et al.  Active image clustering: Seeking constraints from humans to complement algorithms , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Anni Cai,et al.  Enhanced and hierarchical structure algorithm for data imbalance problem in semantic extraction under massive video dataset , 2012, Multimedia Tools and Applications.

[25]  Jason J. Corso,et al.  Spectral active clustering via purification of the K-Nearest neighbor graph , 2012 .

[26]  Qinghua Hu,et al.  Rank Entropy-Based Decision Trees for Monotonic Classification , 2012, IEEE Transactions on Knowledge and Data Engineering.

[27]  Bernadette Bouchon-Meunier,et al.  Improving constrained clustering with active query selection , 2012, Pattern Recognit..

[28]  Nebojsa Jojic,et al.  Active spectral clustering via iterative uncertainty reduction , 2012, KDD.

[29]  Alípio Mário Jorge,et al.  Hierarchical confidence-based active clustering , 2012, SAC '12.

[30]  Yi Yang,et al.  A Multimedia Retrieval Framework Based on Semi-Supervised Ranking and Relevance Feedback , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Sivaraman Balakrishnan,et al.  Efficient Active Algorithms for Hierarchical Clustering , 2012, ICML.

[32]  Tanaya Guha,et al.  Learning Sparse Representations for Human Action Recognition , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  R. Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications. , 2013, IEEE transactions on pattern analysis and machine intelligence.

[34]  Xiaochun Cao,et al.  Video Face Clustering via Constrained Sparse Representation , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[35]  Zan Gao,et al.  Multi-view discriminative and structured dictionary learning with group sparsity for human action recognition , 2015, Signal Process..

[36]  Tat-Seng Chua,et al.  Learning from Collective Intelligence , 2016, ACM Trans. Multim. Comput. Commun. Appl..

[37]  Tat-Seng Chua,et al.  Online Collaborative Learning for Open-Vocabulary Visual Classifiers , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Jason J. Corso,et al.  Active Clustering with Model-Based Uncertainty Reduction , 2014, IEEE transactions on pattern analysis and machine intelligence.

[39]  Joydeep Ghosh,et al.  Semi-Supervised Active Clustering with Weak Oracles , 2017, ArXiv.

[40]  Joydeep Ghosh,et al.  Relaxed Oracles for Semi-Supervised Clustering , 2017, ArXiv.