Robust Sparse Coding via Self-Paced Learning

Sparse coding (SC) is attracting more and more attention due to its comprehensive theoretical studies and its excellent performance in many signal processing applications. However, most existing sparse coding algorithms are nonconvex and are thus prone to becoming stuck into bad local minima, especially when there are outliers and noisy data. To enhance the learning robustness, in this paper, we propose a unified framework named Self-Paced Sparse Coding (SPSC), which gradually include matrix elements into SC learning from easy to complex. We also generalize the self-paced learning schema into different levels of dynamic selection on samples, features and elements respectively. Experimental results on real-world data demonstrate the efficacy of the proposed algorithms.

[1]  Qi Xie,et al.  Self-Paced Learning for Matrix Factorization , 2015, AAAI.

[2]  Zhang Yi,et al.  Underdetermined Blind Source Separation Using Sparse Coding , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Wenjun Zhou,et al.  Multi-Hypergraph Consistent Sparse Coding , 2017, ACM Trans. Intell. Syst. Technol..

[4]  Dacheng Tao,et al.  Multi-view Self-Paced Learning for Clustering , 2015, IJCAI.

[5]  Hang Li,et al.  Reader-Aware Multi-Document Summarization via Sparse Coding , 2015, IJCAI.

[6]  Jaideep Srivastava,et al.  Social network regularized Sparse Linear Model for Top-N recommendation , 2016, Eng. Appl. Artif. Intell..

[7]  Nicu Sebe,et al.  Multi-Paced Dictionary Learning for cross-domain retrieval and recognition , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[8]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Trac D. Tran,et al.  Structured Priors for Sparse-Representation-Based Hyperspectral Image Classification , 2014, IEEE Geoscience and Remote Sensing Letters.

[10]  S. M. García,et al.  2014: , 2020, A Party for Lazarus.

[11]  Qingshan Liu,et al.  Image retrieval via probabilistic hypergraph ranking , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares , 2007, IEEE Journal of Selected Topics in Signal Processing.

[14]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[15]  Allen Y. Yang,et al.  Fast ℓ1-minimization algorithms and an application in robust face recognition: A review , 2010, 2010 IEEE International Conference on Image Processing.

[16]  Chun Chen,et al.  Graph Regularized Sparse Coding for Image Representation , 2011, IEEE Transactions on Image Processing.

[17]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[18]  Junchi Yan,et al.  Self-Paced MultiTask Learning , 2017 .

[19]  Liang-Tien Chia,et al.  Laplacian Sparse Coding, Hypergraph Laplacian Sparse Coding, and Applications , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Dezhong Peng,et al.  Underdetermined Blind Source Separation Using Sparse Coding. , 2016, IEEE transactions on neural networks and learning systems.

[21]  Shuicheng Yan,et al.  Semi-supervised Learning by Sparse Representation , 2009, SDM.

[22]  Baoxin Li,et al.  Discriminative K-SVD for dictionary learning in face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[24]  Sumit Basu,et al.  Teaching Classification Boundaries to Humans , 2013, AAAI.

[25]  Shiguang Shan,et al.  Self-Paced Curriculum Learning , 2015, AAAI.

[26]  Larry S. Davis,et al.  Submodular dictionary learning for sparse coding , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  K. Lange,et al.  Coordinate descent algorithms for lasso penalized regression , 2008, 0803.3876.

[28]  Yang Gao,et al.  Self-paced dictionary learning for image classification , 2012, ACM Multimedia.

[29]  Zhiwu Lu,et al.  Noise-Robust Semi-Supervised Learning by Large-Scale Sparse Coding , 2015, AAAI.

[30]  Jason Weston,et al.  Curriculum learning , 2009, ICML '09.

[31]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[32]  Larry S. Davis,et al.  Label Consistent K-SVD: Learning a Discriminative Dictionary for Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Bing Li,et al.  Multi-View Multi-Instance Learning Based on Joint Sparse Representation and Multi-View Dictionary Learning , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Guangming Shi,et al.  Image restoration via Bayesian structured sparse coding , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[36]  Daphne Koller,et al.  Self-Paced Learning for Latent Variable Models , 2010, NIPS.

[37]  Xiao-Yuan Jing,et al.  Uncorrelated Multi-View Discrimination Dictionary Learning for Recognition , 2014, AAAI.

[38]  David Zhang,et al.  Fisher Discrimination Dictionary Learning for sparse representation , 2011, 2011 International Conference on Computer Vision.

[39]  Jiangping Wang,et al.  Data Clustering by Laplacian Regularized L1-Graph , 2014, AAAI.