Empirical Chaos Processes and Blind Deconvolution

This paper investigates conditions under which certain kinds of systems of bilinear equations have a unique structured solution. In particular, we look at when we can recover vectors $\boldsymbol{w},\boldsymbol{q}$ from observations of the form \[ y_{\ell} = , \quad \ell = 1,\ldots,L, \] where $\boldsymbol{b}_\ell,\boldsymbol{c}_\ell$ are known. We show that if $\boldsymbol{w}\in\mathbb{C}^{M_1}$ and $\boldsymbol{q}\in\mathbb{C}^{M_2}$ are sparse, with no more than $K$ and $N$ nonzero entries, respectively, and the $\boldsymbol{b}_\ell,\boldsymbol{c}_\ell$ are generic, selected as independent Gaussian random vectors, then $\boldsymbol{w},\boldsymbol{q}$ are uniquely determined from \[ L \geq \mathrm{Const}\cdot (K+N)\log^5(M_1M_2) \] such equations with high probability. The key ingredient in our analysis is a uniform probabilistic bound on how far a random process of the form \[Z(\boldsymbol{X}) = \sum_{\ell=1}^L|\boldsymbol{b}_\ell^*\boldsymbol{X}\boldsymbol{c}_\ell|^2 \] deviates from its mean over a set of structured matrices $\boldsymbol{X}\in\mathcal{X}$. As both $\boldsymbol{b}_\ell$ and $\boldsymbol{c}_\ell$ are random, this is a specialized type of $4$th order chaos; we refer to $Z(\boldsymbol{X})$ as an {\em empirical chaos process}. Bounding this process yields a set of general conditions for when the map $\boldsymbol{X}\rightarrow \{\boldsymbol{b}_\ell^*\boldsymbol{X}\boldsymbol{c}_\ell\}_{\ell=1}^L$ is a restricted isometry over the set of matrices $\mathcal{X}$. The conditions are stated in terms of general geometric properties of the set $\mathcal{X}$, and are explicitly computed for the case where $\mathcal{X}$ is the set of matrices that are simultaneously sparse and low rank.

[1]  Miklós Csörgő.  Review: Galen R. Shorack and Jon A. Wellner, Empirical processes with applications to statistics , 1987 .

[2]  Emmanuel J. Candès,et al.  PhaseLift: Exact and Stable Signal Recovery from Magnitude Measurements via Convex Programming , 2011, ArXiv.

[3]  Yanjun Li,et al.  A Unified Framework for Identifiability Analysis in Bilinear Inverse Problems with Applications to Subspace and Sparsity Models , 2015, ArXiv.

[4]  Justin K. Romberg,et al.  Near-Optimal Estimation of Simultaneously Sparse and Low-Rank Matrices from Nested Linear Measurements , 2015, ArXiv.

[5]  Holger Rauhut,et al.  A Mathematical Introduction to Compressive Sensing , 2013, Applied and Numerical Harmonic Analysis.

[6]  Felix Krahmer,et al.  Improved Recovery Guarantees for Phase Retrieval from Coded Diffraction Patterns , 2014, arXiv.org.

[7]  Yuxin Chen,et al.  Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems , 2015, NIPS.

[8]  Justin K. Romberg,et al.  Lifting for Blind Deconvolution in Random Mask Imaging: Identifiability and Convex Relaxation , 2015, SIAM J. Imaging Sci..

[9]  Xiaodong Li,et al.  Solving Quadratic Equations via PhaseLift When There Are About as Many Equations as Unknowns , 2012, Found. Comput. Math..

[10]  M. Talagrand.  Upper and Lower Bounds for Stochastic Processes: Modern Methods and Classical Problems , 2014 .

[11]  Simon Foucart,et al.  Hard Thresholding Pursuit: An Algorithm for Compressive Sensing , 2011, SIAM J. Numer. Anal..

[12]  Peter Jung,et al.  Sparse Model Uncertainties in Compressed Sensing with Application to Convolutions and Sporadic Communication , 2014, ArXiv.

[13]  Yonina C. Eldar,et al.  Phase Retrieval via Matrix Completion , 2011, SIAM Rev..

[14]  Justin K. Romberg,et al.  A tightest convex envelope heuristic to row sparse and rank one matrices , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[15]  D. Pollard.  Empirical Processes: Theory and Applications , 1990 .

[16]  Yanjun Li,et al.  Blind Recovery of Sparse Signals From Subsampled Convolution , 2015, IEEE Transactions on Information Theory.

[17]  Yoram Bresler,et al.  Near Optimal Compressed Sensing of Sparse Rank-One Matrices via Sparse Power Factorization , 2013, ArXiv.

[18]  V. Koltchinskii,et al.  Bounding the smallest singular value of a random matrix without concentration , 2013, 1312.3580.

[19]  Xiaodong Li,et al.  Phase Retrieval from Coded Diffraction Patterns , 2013, 1310.3240.

[20]  Justin K. Romberg,et al.  Blind Deconvolution Using Convex Programming , 2012, IEEE Transactions on Information Theory.

[21]  S. Mendelson,et al.  Empirical processes and random projections , 2005 .

[22]  Yanjun Li,et al.  Identifiability in Blind Deconvolution With Subspace or Sparsity Constraints , 2015, IEEE Transactions on Information Theory.

[23]  Prateek Jain,et al.  Phase Retrieval Using Alternating Minimization , 2013, IEEE Transactions on Signal Processing.

[24]  Massimo Fornasier,et al.  Low-rank Matrix Recovery via Iteratively Reweighted Least Squares Minimization , 2010, SIAM J. Optim..

[25]  Kiryung Lee,et al.  RIP-like Properties in Subsampled Blind Deconvolution , 2015, ArXiv.

[26]  Dan Edidin,et al.  An algebraic characterization of injectivity in phase retrieval , 2013, ArXiv.

[27]  R. Balan,et al.  On signal reconstruction without phase , 2006 .

[28]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[29]  Emmanuel J. Candès,et al.  Tight oracle bounds for low-rank matrix recovery from a minimal number of random measurements , 2010, ArXiv.

[30]  Thomas Strohmer,et al.  Self-calibration and biconvex compressive sensing , 2015, ArXiv.

[31]  B. Recht,et al.  Convex Blind Deconvolution with Random Masks , 2014 .

[32]  Sjoerd Dirksen,et al.  Tail bounds via generic chaining , 2013, ArXiv.

[33]  Yonina C. Eldar,et al.  Uniqueness conditions for low-rank matrix recovery , 2011, Optical Engineering + Applications.

[34]  F. T. Wright,et al.  A Bound on Tail Probabilities for Quadratic Forms in Independent Random Variables , 1971 .

[35]  Justin P. Haldar,et al.  Rank-Constrained Solutions to Linear Matrix Equations Using PowerFactorization , 2009, IEEE Signal Processing Letters.

[36]  Yonina C. Eldar,et al.  Simultaneously Structured Models With Application to Sparse and Low-Rank Matrices , 2012, IEEE Transactions on Information Theory.

[37]  Xiaodong Li,et al.  Rapid, Robust, and Reliable Blind Deconvolution via Nonconvex Optimization , 2016, Applied and Computational Harmonic Analysis.

[38]  M. Rudelson,et al.  On sparse reconstruction from Fourier and Gaussian measurements , 2008 .

[39]  L. Demanet,et al.  Stable Optimizationless Recovery from Phaseless Linear Measurements , 2012, Journal of Fourier Analysis and Applications.

[40]  J. Wellner,et al.  Empirical Processes with Applications to Statistics , 2009 .

[41]  Holger Rauhut,et al.  Suprema of Chaos Processes and the Restricted Isometry Property , 2012, ArXiv.

[42]  B. Carl.  Inequalities of Bernstein-Jackson-type and the degree of compactness of operators in Banach spaces , 1985 .

[43]  E. Giné,et al.  On decoupling, series expansions, and tail behavior of chaos processes , 1993 .

[44]  Yanjun Li,et al.  Identifiability and Stability in Blind Deconvolution Under Minimal Assumptions , 2015, IEEE Transactions on Information Theory.

[45]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[46]  Justin K. Romberg,et al.  Restricted Isometries for Partial Random Circulant Matrices , 2010, ArXiv.

[47]  Yanjun Li,et al.  Identifiability in Blind Deconvolution under Minimal Assumptions , 2015, ArXiv.

[48]  Prateek Jain,et al.  Low-rank matrix completion using alternating minimization , 2012, STOC '13.

[49]  T. Blumensath,et al.  Theory and Applications , 2011 .

[50]  Yang Wang,et al.  Robust sparse phase retrieval made easy , 2014, 1410.5295.

[51]  Felix Krahmer,et al.  Optimal Injectivity Conditions for Bilinear Inverse Problems with Applications to Identifiability of Deconvolution Problems , 2016, SIAM J. Appl. Algebra Geom..

[52]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[53]  Rémi Gribonval,et al.  Recipes for stable linear embeddings from Hilbert spaces to R^m , 2015 .

[54]  Hui Liu,et al.  Recent developments in blind channel equalization: From cyclostationarity to subspaces , 1996, Signal Process..

[55]  S. Mendelson.  Empirical Processes with a Bounded Ψ1 Diameter , 2010 .

[56]  R. Dudley.  The Sizes of Compact Subsets of Hilbert Space and Continuity of Gaussian Processes , 1967 .