Solving Random Systems of Quadratic Equations via Truncated Generalized Gradient Flow

This paper puts forth a novel algorithm, termed \emph{truncated generalized gradient flow} (TGGF), to solve for $\bm{x}\in\mathbb{R}^n/\mathbb{C}^n$ a system of $m$ quadratic equations $y_i=|\langle\bm{a}_i,\bm{x}\rangle|^2$, $i=1,2,\ldots,m$, which even for $\left\{\bm{a}_i\in\mathbb{R}^n/\mathbb{C}^n\right\}_{i=1}^m$ random is known to be \emph{NP-hard} in general. We prove that as soon as the number of equations $m$ is on the order of the number of unknowns $n$, TGGF recovers the solution exactly (up to a global unimodular constant) with high probability and complexity growing linearly with the time required to read the data $\left\{\left(\bm{a}_i;\,y_i\right)\right\}_{i=1}^m$. Specifically, TGGF proceeds in two stages: s1) A novel \emph{orthogonality-promoting} initialization that is obtained with simple power iterations; and, s2) a refinement of the initial estimate by successive updates of scalable \emph{truncated generalized gradient iterations}. The former is in sharp contrast to the existing spectral initializations, while the latter handles the rather challenging nonconvex and nonsmooth \emph{amplitude-based} cost function. Numerical tests demonstrate that: i) The novel orthogonality-promoting initialization method returns more accurate and robust estimates relative to its spectral counterparts; and ii) even with the same initialization, our refinement/truncation outperforms Wirtinger-based alternatives, all corroborating the superior performance of TGGF over state-of-the-art algorithms.

[1]  Yonina C. Eldar,et al.  SIGIBE: Solving random bilinear equations via gradient descent with spectral initialization , 2016, 2016 24th European Signal Processing Conference (EUSIPCO).

[2]  J. Miao,et al.  Extending X-ray crystallography to allow the imaging of noncrystalline materials, cells, and single protein complexes. , 2008, Annual review of physical chemistry.

[3]  J. Miao,et al.  Extending the methodology of X-ray crystallography to allow imaging of micrometre-sized non-crystalline specimens , 1999, Nature.

[4]  Thomas S. Ferguson,et al.  A Representation of the Symmetric Bivariate Cauchy Distribution , 1962 .

[5]  A. Walther The Question of Phase Retrieval in Optics , 1963 .

[6]  G. Simons,et al.  On the theory of elliptically contoured distributions , 1981 .

[7]  Qionghai Dai,et al.  Fourier ptychographic reconstruction using Wirtinger flow optimization. , 2014, Optics express.

[8]  P. Massart,et al.  Adaptive estimation of a quadratic functional by model selection , 2000 .

[9]  Laura Waller,et al.  Experimental robustness of Fourier Ptychography phase retrieval algorithms , 2015, Optics express.

[10]  F. Clarke Optimization And Nonsmooth Analysis , 1983 .

[11]  Yonina C. Eldar,et al.  Phase Retrieval via Matrix Completion , 2011, SIAM Rev..

[12]  R. Balan,et al.  On signal reconstruction without phase , 2006 .

[13]  Yonina C. Eldar,et al.  Phase Retrieval from 1D Fourier Measurements: Convexity, Uniqueness, and Algorithms , 2016, IEEE Transactions on Signal Processing.

[14]  Katta G. Murty,et al.  Some NP-complete problems in quadratic and nonlinear programming , 1987, Math. Program..

[15]  Panos M. Pardalos,et al.  Quadratic programming with one negative eigenvalue is NP-hard , 1991, J. Glob. Optim..

[16]  N. Z. Shor A class of almost-differentiable functions and a minimization method for functions of this class , 1972 .

[17]  Emmanuel J. Candès,et al.  PhaseLift: Exact and Stable Signal Recovery from Magnitude Measurements via Convex Programming , 2011, ArXiv.

[18]  G. Papanicolaou,et al.  Array imaging using intensity-only measurements , 2010 .

[19]  Dan Edidin,et al.  An algebraic characterization of injectivity in phase retrieval , 2013, ArXiv.

[20]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[21]  Anastasios Kyrillidis,et al.  Dropping Convexity for Faster Semi-definite Optimization , 2015, COLT.

[22]  S. Marchesini,et al.  Alternating projection, ptychographic imaging and phase synchronization , 2014, 1402.0550.

[23]  J R Fienup,et al.  Phase retrieval algorithms: a comparison. , 1982, Applied optics.

[24]  J. Gallier Quadratic Optimization Problems , 2020, Linear Algebra and Optimization with Applications to Machine Learning.

[25]  Christopher J. Hillar,et al.  Most Tensor Problems Are NP-Hard , 2009, JACM.

[26]  Franziska Wulf,et al.  Minimization Methods For Non Differentiable Functions , 2016 .

[27]  Zhi-Quan Luo,et al.  Guaranteed Matrix Completion via Non-Convex Factorization , 2014, IEEE Transactions on Information Theory.

[28]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[29]  John Wright,et al.  A Geometric Analysis of Phase Retrieval , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[30]  Dustin G. Mixon,et al.  Saving phase: Injectivity and stability for phase retrieval , 2013, 1302.4618.

[31]  Zhi-Quan Luo,et al.  Semidefinite Relaxation of Quadratic Optimization Problems , 2010, IEEE Signal Processing Magazine.

[32]  H. Sahinoglou,et al.  On phase retrieval of finite-length sequences using the initial time sample , 1991 .

[33]  Yuxin Chen,et al.  Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems , 2015, NIPS.

[34]  Yonina C. Eldar,et al.  Phase Retrieval with Application to Optical Imaging: A contemporary overview , 2015, IEEE Signal Processing Magazine.

[35]  Nikos D. Sidiropoulos,et al.  Phase Retrieval Using Feasible Point Pursuit: Algorithms and Cramér–Rao Bound , 2015, IEEE Transactions on Signal Processing.

[36]  Herbert A. Hauptman,et al.  The phase problem of x-ray crystallography , 1983, Proceedings / Indian Academy of Sciences.

[37]  Prateek Jain,et al.  Phase Retrieval Using Alternating Minimization , 2013, IEEE Transactions on Signal Processing.

[38]  Laurence B. Milstein,et al.  Chernoff-Type Bounds for the Gaussian Error Function , 2011, IEEE Transactions on Communications.

[39]  Rick P. Millane,et al.  Phase retrieval in crystallography and optics , 1990 .

[40]  Gang Wang,et al.  Online reconstruction from big data via compressive censoring , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[41]  Hyoung-Moon Kim,et al.  A Clarification of the Cauchy Distribution , 2014 .

[42]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[43]  Gang Wang,et al.  Adaptive censoring for large-scale regressions , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[44]  Jianqing Fan,et al.  Distributions of angles in random packing on spheres , 2013, J. Mach. Learn. Res..

[45]  Alexandre d'Aspremont,et al.  Phase recovery, MaxCut and complex semidefinite programming , 2012, Math. Program..

[46]  F. Clarke Generalized gradients and applications , 1975 .

[47]  Xiaodong Li,et al.  Solving Quadratic Equations via PhaseLift When There Are About as Many Equations as Unknowns , 2012, Found. Comput. Math..

[48]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[49]  O. Bunk,et al.  Diffractive imaging for periodic samples: retrieving one-dimensional concentration profiles across microfluidic channels. , 2007, Acta crystallographica. Section A, Foundations of crystallography.

[50]  A. Fannjiang,et al.  Phase Retrieval With One or Two Diffraction Patterns by Alternating Projections of the Null Vector , 2015 .

[51]  R. Gerchberg A practical algorithm for the determination of phase from image and diffraction plane pictures , 1972 .

[52]  Ke Wei Solving systems of phaseless equations via Kaczmarz methods: a proof of concept study , 2015 .