Analysis of sparse representations using bi-orthogonal dictionaries

The sparse representation problem of recovering an N dimensional sparse vector x from M <; N linear observations y = Dx given dictionary D is considered. The standard approach is to let the elements of the dictionary be independent and identically distributed (IID) zero-mean Gaussian and minimize the l<sub>1</sub>-norm of x under the constraint y = Dx. In this paper, the performance of l<sub>1</sub>-reconstruction is analyzed, when the dictionary is bi-orthogonal D = [O<sub>1</sub> O<sub>2</sub>], where O<sub>1</sub>, O<sub>2</sub> are independent and drawn uniformly according to the Haar measure on the group of orthogonal M × M matrices. By an application of the replica method, we obtain the critical conditions under which perfect l<sub>1</sub>-recovery is possible with bi-orthogonal dictionaries.

[1]  Koujin Takeda,et al.  Statistical mechanical analysis of compressed sensing utilizing correlated compression matrix , 2010, 2010 IEEE International Symposium on Information Theory.

[2]  Mikko Vehkaperä,et al.  Typical l1-recovery limit of sparse vectors represented by concatenations of random orthogonal matrices , 2012, ArXiv.

[3]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[4]  Dongning Guo,et al.  A single-letter characterization of optimal noisy compressed sensing , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[5]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[6]  Sundeep Rangan,et al.  Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing , 2009, IEEE Transactions on Information Theory.

[7]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[8]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[9]  Michael Elad,et al.  A generalized uncertainty principle and sparse representation in pairs of bases , 2002, IEEE Trans. Inf. Theory.

[10]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[11]  Yoshiyuki Kabashima,et al.  Erratum: A typical reconstruction limit of compressed sensing based on Lp-norm minimization , 2009, ArXiv.

[12]  Toshiyuki Tanaka,et al.  Optimal incorporation of sparsity information by weighted ℓ1 optimization , 2010, 2010 IEEE International Symposium on Information Theory.

[13]  Shlomo Shamai,et al.  Support Recovery With Sparsely Sampled Free Random Matrices , 2011, IEEE Transactions on Information Theory.

[14]  Sundeep Rangan,et al.  A sparsity detection framework for on-off random access channels , 2009, 2009 IEEE International Symposium on Information Theory.

[15]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[16]  Xiaoming Huo,et al.  Sparse image representation via combined transforms , 1999 .

[17]  Allen Y. Yang,et al.  Distributed Sensor Perception via Sparse Representation , 2010, Proceedings of the IEEE.

[18]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.