Isotropically random orthogonal matrices: Performance of LASSO and minimum conic singular values

Recently, the precise performance of the Generalized LASSO algorithm for recovering structured signals from compressed noisy measurements, obtained via i.i.d. Gaussian matrices, has been characterized. The analysis is based on a framework introduced by Stojnic and heavily relies on the use of Gordon's Gaussian min-max theorem (GMT), a comparison principle on Gaussian processes. As a result, corresponding characterizations for other ensembles of measurement matrices have not been developed. In this work, we analyze the corresponding performance of the ensemble of isotropically random orthogonal (i.r.o.) measurements. We consider the constrained version of the Generalized LASSO and derive a sharp characterization of its normalized squared error in the large-system limit. When compared to its Gaussian counterpart, our result analytically confirms the superiority in performance of the i.r.o. ensemble. Our second result, derives an asymptotic lower bound on the minimum conic singular values of i.r.o. matrices. This bound is larger than the corresponding bound on Gaussian matrices. To prove our results we express i.r.o. matrices in terms of Gaussians and show that, with some modifications, the GMT framework is still applicable.

[1]  Mihailo Stojnic,et al.  A framework to characterize performance of LASSO algorithms , 2013, ArXiv.

[2]  Mikko Vehkaperä,et al.  Analysis of Regularized LS Reconstruction and Random Matrix Ensembles in Compressed Sensing , 2013, IEEE Transactions on Information Theory.

[3]  Joel A. Tropp,et al.  A comparison principle for functions of a uniformly random subspace , 2011, 1102.0534.

[4]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[5]  Joel A. Tropp,et al.  Living on the edge: phase transitions in convex programs with random data , 2013, 1303.6672.

[6]  Christos Thrampoulidis,et al.  Asymptotically exact error analysis for the generalized equation-LASSO , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[7]  Christos Thrampoulidis,et al.  The squared-error of generalized LASSO: A precise analysis , 2013, 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[8]  Joel A. Tropp,et al.  Living on the edge: A geometric theory of phase transitions in convex optimization , 2013, ArXiv.

[9]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[10]  Christos Thrampoulidis,et al.  Asymptotically Exact Error Analysis for the Generalized $\ell_2^2$-LASSO , 2015, ISIT 2015.

[11]  Y. Gordon On Milman's inequality and random subspaces which escape through a mesh in ℝ n , 1988 .

[12]  Sergio Verdú,et al.  Optimal Phase Transitions in Compressed Sensing , 2011, IEEE Transactions on Information Theory.

[13]  Tiefeng Jiang,et al.  How many entries of a typical orthogonal matrix can be approximated by independent normals , 2006 .

[14]  Chau Yuen,et al.  On Sparse Vector Recovery Performance in Structurally Orthogonal Matrices via LASSO , 2014, IEEE Transactions on Signal Processing.

[15]  W. Marsden I and J , 2012 .

[16]  Andrea Montanari,et al.  The Noise-Sensitivity Phase Transition in Compressed Sensing , 2010, IEEE Transactions on Information Theory.

[17]  Rina Foygel,et al.  Corrupted Sensing: Novel Guarantees for Separating Structured Signals , 2013, IEEE Transactions on Information Theory.

[18]  Babak Hassibi,et al.  A case for orthogonal measurements in linear inverse problems , 2014, 2014 IEEE International Symposium on Information Theory.

[19]  R. Gill,et al.  Cox's regression model for counting processes: a large sample study : (preprint) , 1982 .

[20]  W. Newey,et al.  Large sample estimation and hypothesis testing , 1986 .

[21]  Christos Thrampoulidis,et al.  Estimating structured signals in sparse noise: A precise noise sensitivity analysis , 2014, 2014 52nd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[22]  Dimitri P. Bertsekas,et al.  Convex Analysis and Optimization , 2003 .

[23]  Andrea Montanari,et al.  The LASSO Risk for Gaussian Matrices , 2010, IEEE Transactions on Information Theory.

[24]  丸山 徹 Convex Analysisの二,三の進展について , 1977 .

[25]  Christos Thrampoulidis,et al.  A Tight Version of the Gaussian min-max theorem in the Presence of Convexity , 2014, ArXiv.

[26]  Christos Thrampoulidis,et al.  Precise error analysis of the LASSO , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).