Phase Transition of Convex Programs for Linear Inverse Problems with Multiple Prior Constraints

A sharp phase transition emerges in convex programs when solving the linear inverse problem, which aims to recover a structured signal from its linear measurements. This paper studies this phenomenon in theory under Gaussian random measurements. Different from previous studies, in this paper, we consider convex programs with multiple prior constraints. These programs are encountered in many cases, for example, when the signal is sparse and its $\ell_2$ norm is known beforehand, or when the signal is sparse and non-negative simultaneously. Given such a convex program, to analyze its phase transition, we introduce a new set and a new cone, called the prior restricted set and prior restricted cone, respectively. Our results reveal that the phase transition of a convex problem occurs at the statistical dimension of its prior restricted cone. Moreover, to apply our theoretical results in practice, we present two recipes to accurately estimate the statistical dimension of the prior restricted cone. These two recipes work under different conditions, and we give a detailed analysis for them. To further illustrate our results, we apply our theoretical results and the estimation recipes to study the phase transition of two specific problems, and obtain computable formulas for the statistical dimension and related error bounds. Simulations are provided to demonstrate our results.

[1]  David L. Donoho,et al.  Exponential Bounds Implying Construction of Compressed Sensing Matrices, Error-Correcting Codes, and Neighborly Polytopes by Random Sampling , 2010, IEEE Transactions on Information Theory.

[2]  Andrea Montanari,et al.  Universality in Polytope Phase Transitions and Message Passing Algorithms , 2012, ArXiv.

[3]  Andrea Montanari,et al.  Accurate Prediction of Phase Transitions in Compressed Sensing via a Connection to Minimax Denoising , 2011, IEEE Transactions on Information Theory.

[4]  Y. Gordon.  On Milman's inequality and random subspaces which escape through a mesh in ℝ n , 1988 .

[5]  R. Bartle.  The elements of integration and Lebesgue measure , 1995 .

[6]  Joel A. Tropp,et al.  Universality laws for randomized dimension reduction, with applications , 2015, ArXiv.

[7]  Boris S. Mordukhovich,et al.  An Easy Path to Convex Analysis and Applications , 2013, Synthesis Lectures on Mathematics & Statistics.

[8]  Joel A. Tropp,et al.  Convex recovery of a structured signal from independent random linear measurements , 2014, ArXiv.

[9]  M. Spivak.  Calculus On Manifolds: A Modern Approach To Classical Theorems Of Advanced Calculus , 2019 .

[10]  Babak Hassibi,et al.  Asymptotically Exact Denoising in Relation to Compressed Sensing , 2013, ArXiv.

[11]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[12]  David L. Donoho,et al.  Counting the Faces of Randomly-Projected Hypercubes and Orthants, with Applications , 2008, Discret. Comput. Geom..

[13]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[14]  D. Donoho,et al.  Neighborliness of randomly projected simplices in high dimensions. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Andrea Montanari,et al.  The phase transition of matrix recovery from Gaussian measurements matches the minimax MSE of matrix denoising , 2013, Proceedings of the National Academy of Sciences.

[16]  M. Rudelson,et al.  On sparse reconstruction from Fourier and Gaussian measurements , 2008 .

[17]  R. Nowak,et al.  Compressed Sensing for Networked Data , 2008, IEEE Signal Processing Magazine.

[18]  Joel A. Tropp,et al.  Living on the edge: phase transitions in convex programs with random data , 2013, 1303.6672.

[19]  W. Rudin.  Principles of mathematical analysis , 1964 .

[20]  D. Donoho,et al.  Counting faces of randomly-projected polytopes when the projection radically lowers dimension , 2006, math/0607364.

[21]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.