A Tighter Bound on the Uniqueness of Sparsely-Used Dictionaries

In this paper, we consider the sparsely used dictionary learning problem and focus on the case that the dictionary is square with arbitrary entries and the coefficient matrix is a sparse with random entries, which is first proposed and studied by Wang and Spielman et al.[1], [2]. We improve the theoretical results with a tighter bounded uniqueness theorem, which says that O(n) samples are sufficient to uniquely determine the decomposition with probability one when given assumption is satisfied, and the proof is given.

[1]  Julien Mairal,et al.  Proximal Methods for Hierarchical Sparse Coding , 2010, J. Mach. Learn. Res..

[2]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[3]  Shie Mannor,et al.  The Sample Complexity of Dictionary Learning , 2010, COLT.

[4]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[5]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[6]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[7]  Rémi Gribonval,et al.  Sparse decompositions in "incoherent" dictionaries , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[8]  Francis R. Bach,et al.  Structured Variable Selection with Sparsity-Inducing Norms , 2009, J. Mach. Learn. Res..

[9]  H. Chernoff.  A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations , 1952 .

[10]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[11]  D. Donoho,et al.  Basis pursuit , 1994, Proceedings of 1994 28th Asilomar Conference on Signals, Systems and Computers.

[12]  Alexander G. Gray,et al.  On the Sample Complexity of Predictive Sparse Coding , 2012, ArXiv.

[13]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[14]  Ravi Ramamoorthi,et al.  Analytic PCA Construction for Theoretical Analysis of Lighting Variability in Images of a Lambertian Object , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[16]  Emmanuel J. Candès,et al.  A Geometric Analysis of Subspace Clustering with Outliers , 2011, ArXiv.

[17]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[18]  Huan Wang,et al.  Exact Recovery of Sparsely-Used Dictionaries , 2012, COLT.

[19]  René Vidal,et al.  Motion segmentation via robust subspace separation in the presence of outlying, incomplete, or corrupted trajectories , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Rémi Gribonval,et al.  Sample Complexity of Dictionary Learning and Other Matrix Factorizations , 2013, IEEE Transactions on Information Theory.

[21]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[22]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[23]  Huan Wang,et al.  Dictionary Learning: Algorithms and Analysis , 2013 .