Non-linear dictionary learning with partially labeled data

While recent techniques for discriminative dictionary learning have demonstrated tremendous success in image analysis applications, their performance is often limited by the amount of labeled data available for training. Even though labeling images is difficult, it is relatively easy to collect unlabeled images either by querying the web or from public datasets. Using the kernel method, we propose a non-linear discriminative dictionary learning technique which utilizes both labeled and unlabeled data for learning dictionaries in the high-dimensional feature space. Furthermore, we show how this method can be extended for ambiguously labeled classification problem where each training sample has multiple labels and only one of them is correct. Extensive evaluation on existing datasets demonstrates that the proposed method performs significantly better than state of the art dictionary learning approaches when unlabeled images are available for training. HighlightsA dictionary learning method that utilizes labeled and unlabeled data is proposed.Using kernel trick, the proposed formulation is extended to the non-linear case.An efficient optimization procedure is proposed for solving this non-linear problem.Each training sample can have multiple labels and only one of them is correct.

[1]  Guillermo Sapiro,et al.  Classification and clustering via dictionary learning with structured incoherence and shared features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Bernhard Schölkopf,et al.  Semi-Supervised Learning (Adaptive Computation and Machine Learning) , 2006 .

[3]  Rama Chellappa,et al.  Dictionary-Based Face Recognition from Video , 2012, ECCV.

[4]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Guillermo Sapiro,et al.  Dictionary learning and sparse coding for unsupervised clustering , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[6]  Ben Taskar,et al.  Learning from Partial Labels , 2011, J. Mach. Learn. Res..

[7]  Rama Chellappa,et al.  Design of Non-Linear Discriminative Dictionaries for Image Classification , 2012, ACCV.

[8]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[9]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[10]  Aapo Hyvärinen,et al.  DirectLiNGAM: A Direct Method for Learning a Linear Non-Gaussian Structural Equation Model , 2011, J. Mach. Learn. Res..

[11]  Marc'Aurelio Ranzato,et al.  Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  René Vidal,et al.  Sparse subspace clustering , 2009, CVPR.

[13]  Lorenzo Rosasco,et al.  Iterative Projection Methods for Structured Sparsity Regularization , 2009 .

[14]  Michael Elad,et al.  On the Role of Sparse and Redundant Representations in Image Processing , 2010, Proceedings of the IEEE.

[15]  Larry S. Davis,et al.  Learning a discriminative dictionary for sparse coding via label consistent K-SVD , 2011, CVPR 2011.

[16]  Rama Chellappa,et al.  Design of Non-Linear Kernel Dictionaries for Object Recognition , 2013, IEEE Transactions on Image Processing.

[17]  Rama Chellappa,et al.  Sparse Embedding: A Framework for Sparsity Promoting Dimensionality Reduction , 2012, ECCV.

[18]  Baoxin Li,et al.  Discriminative K-SVD for dictionary learning in face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[20]  Yixin Chen,et al.  Automatic Feature Decomposition for Single View Co-training , 2011, ICML.

[21]  Rama Chellappa,et al.  Separability-based multiscale basis selection and feature extraction for signal and image classification , 1998, IEEE Trans. Image Process..

[22]  Rama Chellappa,et al.  Dictionary-Based Face Recognition Under Variable Lighting and Pose , 2012, IEEE Transactions on Information Forensics and Security.

[23]  David Zhang,et al.  Fisher Discrimination Dictionary Learning for sparse representation , 2011, 2011 International Conference on Computer Vision.

[24]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[25]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[26]  Ke Huang,et al.  Sparse Representation for Signal Classification , 2006, NIPS.

[27]  Rama Chellappa,et al.  In-Plane Rotation and Scale Invariant Clustering Using Dictionaries , 2013, IEEE Transactions on Image Processing.

[28]  Rama Chellappa,et al.  Learning discriminative dictionaries with partially labeled data , 2012, 2012 19th IEEE International Conference on Image Processing.

[29]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[30]  Jean Ponce,et al.  Task-Driven Dictionary Learning , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  S. Sathiya Keerthi,et al.  Large scale semi-supervised linear SVMs , 2006, SIGIR.

[32]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[33]  Rama Chellappa,et al.  Dictionary Learning from Ambiguously Labeled Data , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Larry S. Davis,et al.  Online Semi-Supervised Discriminative Dictionary Learning for Sparse Representation , 2012, ACCV.

[35]  Guillermo Sapiro,et al.  Sparse representations for image classification: learning discriminative and reconstructive non-parametric dictionaries , 2008 .

[36]  B. Taskar,et al.  Learning from ambiguously labeled images , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Philip N. Klein,et al.  Recognition of Shapes by Editing Shock Graphs , 2001, ICCV.

[38]  Rama Chellappa,et al.  Sparse Representations, Compressive Sensing and dictionaries for pattern recognition , 2011, The First Asian Conference on Pattern Recognition.