Solving Systems of Random Quadratic Equations via Truncated Amplitude Flow

This paper presents a new algorithm, termed <italic>truncated amplitude flow</italic> (TAF), to recover an unknown vector <inline-formula> <tex-math notation="LaTeX">$ {x}$ </tex-math></inline-formula> from a system of quadratic equations of the form <inline-formula> <tex-math notation="LaTeX">$y_{i}=|\langle {a}_{i}, {x}\rangle |^{2}$ </tex-math></inline-formula>, where <inline-formula> <tex-math notation="LaTeX">$ {a}_{i}$ </tex-math></inline-formula>’s are given random measurement vectors. This problem is known to be <italic>NP-hard</italic> in general. We prove that as soon as the number of equations is on the order of the number of unknowns, TAF recovers the solution exactly (up to a global unimodular constant) with high probability and complexity growing linearly with both the number of unknowns and the number of equations. Our TAF approach adapts the <italic>amplitude-based</italic> empirical loss function and proceeds in two stages. In the first stage, we introduce an <italic>orthogonality-promoting</italic> initialization that can be obtained with a few power iterations. Stage two refines the initial estimate by successive updates of scalable <italic>truncated generalized gradient iterations</italic>, which are able to handle the rather challenging nonconvex and nonsmooth amplitude-based objective function. In particular, when vectors <inline-formula> <tex-math notation="LaTeX">$ {x}$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">${a}_{i}$ </tex-math></inline-formula>’s are real valued, our gradient truncation rule provably eliminates erroneously estimated signs with high probability to markedly improve upon its untruncated version. Numerical tests using synthetic data and real images demonstrate that our initialization returns more accurate and robust estimates relative to spectral initializations. Furthermore, even under the same initialization, the proposed amplitude-based refinement outperforms existing Wirtinger flow variants, corroborating the superior performance of TAF over state-of-the-art algorithms.

[1]  John Wright,et al.  A Geometric Analysis of Phase Retrieval , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[2]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[3]  Jianqing Fan,et al.  Distributions of angles in random packing on spheres , 2013, J. Mach. Learn. Res..

[4]  Alexandre d'Aspremont,et al.  Phase recovery, MaxCut and complex semidefinite programming , 2012, Math. Program..

[5]  Gang Wang,et al.  Solving large-scale systems of random quadratic equations via stochastic truncated amplitude flow , 2016, 2017 25th European Signal Processing Conference (EUSIPCO).

[6]  J R Fienup,et al.  Phase retrieval algorithms: a comparison. , 1982, Applied optics.

[7]  P. Massart,et al.  Adaptive estimation of a quadratic functional by model selection , 2000 .

[8]  Babak Hassibi,et al.  Sparse Phase Retrieval: Uniqueness Guarantees and Recovery Algorithms , 2013, IEEE Transactions on Signal Processing.

[9]  Yuejie Chi,et al.  Reshaped Wirtinger Flow and Incremental Algorithm for Solving Quadratic System of Equations , 2016 .

[10]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[11]  Prateek Jain,et al.  Phase Retrieval Using Alternating Minimization , 2013, IEEE Transactions on Signal Processing.

[12]  Herbert A. Hauptman,et al.  The phase problem of x-ray crystallography , 1983, Proceedings / Indian Academy of Sciences.

[13]  Rick P. Millane,et al.  Phase retrieval in crystallography and optics , 1990 .

[14]  Gang Wang,et al.  Online reconstruction from big data via compressive censoring , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[15]  Yonina C. Eldar,et al.  Phase Retrieval from 1D Fourier Measurements: Convexity, Uniqueness, and Algorithms , 2016, IEEE Transactions on Signal Processing.

[16]  Dan Edidin,et al.  An algebraic characterization of injectivity in phase retrieval , 2013, ArXiv.

[17]  Gang Wang,et al.  Solving Random Systems of Quadratic Equations via Truncated Generalized Gradient Flow , 2016, NIPS.

[18]  Xiaodong Li,et al.  Rapid, Robust, and Reliable Blind Deconvolution via Nonconvex Optimization , 2016, Applied and Computational Harmonic Analysis.

[19]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[20]  J. Miao,et al.  Extending X-ray crystallography to allow the imaging of noncrystalline materials, cells, and single protein complexes. , 2008, Annual review of physical chemistry.

[21]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[22]  Anastasios Kyrillidis,et al.  Dropping Convexity for Faster Semi-definite Optimization , 2015, COLT.

[23]  S. Marchesini,et al.  Alternating projection, ptychographic imaging and phase synchronization , 2014, 1402.0550.

[24]  Pengwen Chen,et al.  Fourier Phase Retrieval with a Single Mask by Douglas-Rachford Algorithm , 2015, Applied and computational harmonic analysis.

[25]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, 2009 IEEE International Symposium on Information Theory.

[26]  Feng Ruan,et al.  Solving (most) of a set of quadratic equalities: Composite optimization for robust phase retrieval , 2017, Information and Inference: A Journal of the IMA.

[27]  G. Papanicolaou,et al.  Array imaging using intensity-only measurements , 2010 .

[28]  Xiaodong Li,et al.  Phase Retrieval from Coded Diffraction Patterns , 2013, 1310.3240.

[29]  Gang Wang,et al.  Solving Most Systems of Random Quadratic Equations , 2017, NIPS.

[30]  F. Clarke Generalized gradients and applications , 1975 .

[31]  Xiaodong Li,et al.  Solving Quadratic Equations via PhaseLift When There Are About as Many Equations as Unknowns , 2012, Found. Comput. Math..

[32]  Sujay Sanghavi,et al.  The Local Convexity of Solving Systems of Quadratic Equations , 2015, 1506.07868.

[33]  Yonina C. Eldar,et al.  GESPAR: Efficient Phase Retrieval of Sparse Signals , 2013, IEEE Transactions on Signal Processing.

[34]  N. Z. Shor A class of almost-differentiable functions and a minimization method for functions of this class , 1972 .

[35]  Laura Waller,et al.  Experimental robustness of Fourier Ptychography phase retrieval algorithms , 2015, Optics express.

[36]  F. Clarke Optimization And Nonsmooth Analysis , 1983 .

[37]  Franziska Wulf,et al.  Minimization Methods For Non Differentiable Functions , 2016 .

[38]  Zhi-Quan Luo,et al.  Guaranteed Matrix Completion via Non-Convex Factorization , 2014, IEEE Transactions on Information Theory.

[39]  Nikos D. Sidiropoulos,et al.  Inexact Alternating Optimization for Phase Retrieval in the Presence of Outliers , 2016, IEEE Transactions on Signal Processing.

[40]  Nikos D. Sidiropoulos,et al.  Phase Retrieval Using Feasible Point Pursuit: Algorithms and Cramér–Rao Bound , 2015, IEEE Transactions on Signal Processing.

[41]  Yonina C. Eldar,et al.  Phase Retrieval via Matrix Completion , 2011, SIAM Rev..

[42]  R. Balan,et al.  On signal reconstruction without phase , 2006 .

[43]  Hyoung-Moon Kim,et al.  A Clarification of the Cauchy Distribution , 2014 .

[44]  Zhang Fe Phase retrieval from coded diffraction patterns , 2015 .

[45]  J R Fienup,et al.  Reconstruction of an object from the modulus of its Fourier transform. , 1978, Optics letters.

[46]  Yingbin Liang,et al.  Median-Truncated Nonconvex Approach for Phase Retrieval With Outliers , 2016, IEEE Transactions on Information Theory.

[47]  Laurence B. Milstein,et al.  Chernoff-Type Bounds for the Gaussian Error Function , 2011, IEEE Transactions on Communications.

[48]  Bernhard G. Bodmann,et al.  Stable phase retrieval with low-redundancy frames , 2013, Adv. Comput. Math..

[49]  Yingbin Liang,et al.  Provable Non-convex Phase Retrieval with Outliers: Median TruncatedWirtinger Flow , 2016, ICML.

[50]  Yonina C. Eldar,et al.  SIGIBE: Solving random bilinear equations via gradient descent with spectral initialization , 2016, 2016 24th European Signal Processing Conference (EUSIPCO).

[51]  Thomas S. Ferguson,et al.  A Representation of the Symmetric Bivariate Cauchy Distribution , 1962 .

[52]  A. Walther The Question of Phase Retrieval in Optics , 1963 .

[53]  Gang Wang,et al.  Adaptive censoring for large-scale regressions , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[54]  Yonina C. Eldar,et al.  STFT Phase Retrieval: Uniqueness Guarantees and Recovery Algorithms , 2015, IEEE Journal of Selected Topics in Signal Processing.

[55]  EDWARD M. HOFSTETTER,et al.  Construction of time-limited functions with specified autocorrelation functions , 1964, IEEE Trans. Inf. Theory.

[56]  J. Miao,et al.  Extending the methodology of X-ray crystallography to allow imaging of micrometre-sized non-crystalline specimens , 1999, Nature.

[57]  Qionghai Dai,et al.  Fourier ptychographic reconstruction using Wirtinger flow optimization. , 2014, Optics express.

[58]  Dustin G. Mixon,et al.  Saving phase: Injectivity and stability for phase retrieval , 2013, 1302.4618.

[59]  Zhi-Quan Luo,et al.  Semidefinite Relaxation of Quadratic Optimization Problems , 2010, IEEE Signal Processing Magazine.

[60]  H. Sahinoglou,et al.  On phase retrieval of finite-length sequences using the initial time sample , 1991 .

[61]  Yuxin Chen,et al.  Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems , 2015, NIPS.

[62]  Yonina C. Eldar,et al.  Phase Retrieval with Application to Optical Imaging: A contemporary overview , 2015, IEEE Signal Processing Magazine.

[63]  G. Simons,et al.  On the theory of elliptically contoured distributions , 1981 .

[64]  J. Gallier Quadratic Optimization Problems , 2020, Linear Algebra and Optimization with Applications to Machine Learning.

[65]  Christopher J. Hillar,et al.  Most Tensor Problems Are NP-Hard , 2009, JACM.

[66]  Herbert A. Hauptman The phase problem of X-ray crystallography , 1991 .

[67]  Gang Wang,et al.  Solving Almost all Systems of Random Quadratic Equations , 2017, NIPS 2017.

[68]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[69]  O. Bunk,et al.  Diffractive imaging for periodic samples: retrieving one-dimensional concentration profiles across microfluidic channels. , 2007, Acta crystallographica. Section A, Foundations of crystallography.

[70]  A. Fannjiang,et al.  Phase Retrieval With One or Two Diffraction Patterns by Alternating Projections of the Null Vector , 2015 .

[71]  Yonina C. Eldar,et al.  Phase Retrieval: Stability and Recovery Guarantees , 2012, ArXiv.

[72]  R. Gerchberg A practical algorithm for the determination of phase from image and diffraction plane pictures , 1972 .

[73]  A. Fannjiang,et al.  Phase Retrieval with One or Two Diffraction Patterns by Alternating Projections with the Null Initialization , 2015, 1510.07379.

[74]  Yonina C. Eldar,et al.  Sparse Phase Retrieval from Short-Time Fourier Measurements , 2014, IEEE Signal Processing Letters.

[75]  Ke Wei Solving systems of phaseless equations via Kaczmarz methods: a proof of concept study , 2015 .

[76]  Gang Wang,et al.  Sparse Phase Retrieval via Truncated Amplitude Flow , 2016, IEEE Transactions on Signal Processing.

[77]  Max Simchowitz,et al.  Low-rank Solutions of Linear Matrix Equations via Procrustes Flow , 2015, ICML.

[78]  Yonina C. Eldar,et al.  Phase Retrieval: An Overview of Recent Developments , 2015, ArXiv.

[79]  Katta G. Murty,et al.  Some NP-complete problems in quadratic and nonlinear programming , 1987, Math. Program..

[80]  Panos M. Pardalos,et al.  Quadratic programming with one negative eigenvalue is NP-hard , 1991, J. Glob. Optim..

[81]  Emmanuel J. Candès,et al.  PhaseLift: Exact and Stable Signal Recovery from Magnitude Measurements via Convex Programming , 2011, ArXiv.