A Joint Optimization Framework of Sparse Coding and Discriminative Clustering

Many clustering methods highly depend on extracted features. In this paper, we propose a joint optimization framework in terms of both feature extraction and discriminative clustering. We utilize graph regularized sparse codes as the features, and formulate sparse coding as the constraint for clustering. Two cost functions are developed based on entropy-minimization and maximum-margin clustering principles, respectively, as the objectives to be minimized. Solving such a bi-level optimization mutually reinforces both sparse coding and clustering steps. Experiments on several benchmark datasets verify remarkable performance improvements led by the proposed joint optimization.

[1]  Dale Schuurmans,et al.  Unsupervised and Semi-Supervised Multi-Class Support Vector Machines , 2005, AAAI.

[2]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[3]  Dale Schuurmans,et al.  Maximum Margin Clustering , 2004, NIPS.

[4]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[5]  Chih-Jen Lin,et al.  A Study on L2-Loss (Squared Hinge-Loss) Multiclass SVM , 2013, Neural Computation.

[6]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[7]  YanShuicheng,et al.  Learning with l1-graph for image analysis , 2010 .

[8]  Dima Damen,et al.  Recognizing linked events: Searching the space of feasible explanations , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Volker Roth,et al.  Feature Selection in Clustering Problems , 2003, NIPS.

[10]  Ibon Saratxaga,et al.  Detection of synthetic speech for the problem of imposture , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[11]  Bo Dai,et al.  Minimum Conditional Entropy Clustering: A Discriminative Framework for Clustering , 2010, ACML.

[12]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[13]  Shuicheng Yan,et al.  On a Theory of Nonparametric Pairwise Similarity for Clustering: Connecting Clustering to Classification , 2014, NIPS.

[14]  Gérard Govaert,et al.  Assessing a Mixture Model for Clustering with the Integrated Completed Likelihood , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Thomas S. Huang,et al.  Semisupervised Hyperspectral Classification Using Task-Driven Dictionary Learning With Laplacian Regularization , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Jiawei Han,et al.  Regularized l1-Graph for Data Clustering , 2014, BMVC.

[17]  Fei Wang,et al.  Efficient Maximum Margin Clustering via Cutting Plane Algorithm , 2008, SDM.

[18]  Tao Jiang,et al.  Minimum entropy clustering and applications to gene expression analysis , 2004, Proceedings. 2004 IEEE Computational Systems Bioinformatics Conference, 2004. CSB 2004..

[19]  Thomas S. Huang,et al.  Bilevel sparse coding for coupled feature spaces , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[21]  Shuicheng Yan,et al.  Learning With $\ell ^{1}$-Graph for Image Analysis , 2010, IEEE Transactions on Image Processing.

[22]  Chun Chen,et al.  Graph Regularized Sparse Coding for Image Representation , 2011, IEEE Transactions on Image Processing.

[23]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[24]  Ivor W. Tsang,et al.  Tighter and Convex Maximum Margin Clustering , 2009, AISTATS.

[25]  BiernackiChristophe,et al.  Assessing a Mixture Model for Clustering with the Integrated Completed Likelihood , 2000 .

[26]  David Barber,et al.  Kernelized Infomax Clustering , 2005, NIPS.

[27]  Jiangping Wang,et al.  Data Clustering by Laplacian Regularized L1-Graph , 2014, AAAI.

[28]  Jean Ponce,et al.  Task-Driven Dictionary Learning , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Guillermo Sapiro,et al.  Dictionary learning and sparse coding for unsupervised clustering , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.