A Geometric Analysis of Phase Retrieval

Can we recover a complex signal from its Fourier magnitudes? More generally, given a set of m measurements, $$y_k = \left| \varvec{a}_k^* \varvec{x} \right| $$yk=ak∗x for $$k = 1, \ldots , m$$k=1,…,m, is it possible to recover $$\varvec{x} \in \mathbb C^n$$x∈Cn (i.e., length-n complex vector)? This generalized phase retrieval (GPR) problem is a fundamental task in various disciplines and has been the subject of much recent investigation. Natural nonconvex heuristics often work remarkably well for GPR in practice, but lack clear theoretic explanations. In this paper, we take a step toward bridging this gap. We prove that when the measurement vectors $$\varvec{a}_k$$ak’s are generic (i.i.d. complex Gaussian) and numerous enough ($$m \ge C n \log ^3 n$$m≥Cnlog3n), with high probability, a natural least-squares formulation for GPR has the following benign geometric structure: (1) There are no spurious local minimizers, and all global minimizers are equal to the target signal $$\varvec{x}$$x, up to a global phase, and (2) the objective function has a negative directional curvature around each saddle point. This structure allows a number of iterative optimization methods to efficiently find a global minimizer, without special initialization. To corroborate the claim, we describe and analyze a second-order trust-region algorithm.

[1]  Hans Reichenbach,et al.  Philosophic foundations of quantum mechanics , 1945 .

[2]  Edmund Taylor Whittaker.  Philosophic Foundations of Quantum Mechanics , 1946, Nature.

[3]  A. Walther.  The Question of Phase Retrieval in Optics , 1963 .

[4]  R. Gerchberg.  A practical algorithm for the determination of phase from image and diffraction plane pictures , 1972 .

[5]  Donald Goldfarb,et al.  Curvilinear path steplength algorithms for minimization which use directions of negative curvature , 1980, Math. Program..

[6]  J R Fienup,et al.  Phase retrieval algorithms: a comparison. , 1982, Applied optics.

[7]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[8]  Richard Zippel,et al.  Proving Polynomial-Time for Sphere-Constrained Quadratic Programming , 1990 .

[9]  Rick P. Millane,et al.  Phase retrieval in crystallography and optics , 1990 .

[10]  G. Stewart,et al.  Matrix Perturbation Theory , 1990 .

[11]  Yinyu Ye,et al.  On affine scaling algorithms for nonconvex quadratic programming , 1992, Math. Program..

[12]  Robert W. Harrison,et al.  Phase problem in crystallography , 1993 .

[13]  Alberto Maria Segre.  The Ninth International Conference on Machine Learning , 1993, AI Mag..

[14]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[15]  Franz Rendl,et al.  A semidefinite framework for trust region subproblems with applications to large scale minimization , 1997, Math. Program..

[16]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[17]  Michael I. Jordan,et al.  Learning with Mixtures of Trees , 2001, J. Mach. Learn. Res..

[18]  K. Roberts,et al.  Thesis , 2002 .

[19]  J. Miao,et al.  High resolution 3D x-ray diffraction microscopy. , 2002, Physical review letters.

[20]  Henry Wolkowicz,et al.  The trust region subproblem and semidefinite programming , 2004, Optim. Methods Softw..

[21]  Ken Kreutz-Delgado,et al.  The Complex Gradient Operator and the CR-Calculus ECE275A - Lecture Supplement - Fall 2005 , 2009, 0906.4835.

[22]  J. Corbett.  The pauli problem, state reconstruction and quantum-real numbers , 2006 .

[23]  R. Balan,et al.  On signal reconstruction without phase , 2006 .

[24]  Yurii Nesterov,et al.  Cubic regularization of Newton method and its global performance , 2006, Math. Program..

[25]  Robert E. Mahony,et al.  Optimization Algorithms on Matrix Manifolds , 2007 .

[26]  O. Bunk,et al.  Diffractive imaging for periodic samples: retrieving one-dimensional concentration profiles across microfluidic channels. , 2007, Acta crystallographica. Section A, Foundations of crystallography.

[27]  Pierre-Antoine Absil,et al.  Trust-Region Methods on Riemannian Manifolds , 2007, Found. Comput. Math..

[28]  R. Balan,et al.  Painless Reconstruction from Magnitudes of Frame Coefficients , 2009 .

[29]  Levent Tunçel,et al.  Optimization algorithms on matrix manifolds , 2009, Math. Comput..

[30]  Radu V. Balan,et al.  On signal reconstruction from its spectrogram , 2010, 2010 44th Annual Conference on Information Sciences and Systems (CISS).

[31]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, 2009 IEEE International Symposium on Information Theory.

[32]  G. Papanicolaou,et al.  Array imaging using intensity-only measurements , 2010 .

[33]  Emmanuel J. Candès,et al.  PhaseLift: Exact and Stable Signal Recovery from Magnitude Measurements via Convex Programming , 2011, ArXiv.

[34]  Allen Y. Yang,et al.  CPRL -- An Extension of Compressive Sensing to the Phase Retrieval Problem , 2012, NIPS.

[35]  S. Sastry,et al.  Compressive Phase Retrieval From Squared Output Measurements Via Semidefinite Programming , 2011, 1111.6323.

[36]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[37]  Nicholas I. M. Gould,et al.  Complexity bounds for second-order optimality in unconstrained optimization , 2012, J. Complex..

[38]  Yonina C. Eldar,et al.  Efficient phase retrieval of sparse signals , 2012, 2012 IEEE 27th Convention of Electrical and Electronics Engineers in Israel.

[39]  Xiaodong Li,et al.  Phase Retrieval from Coded Diffraction Patterns , 2013, 1310.3240.

[40]  Felix Krahmer,et al.  A Partial Derandomization of PhaseLift Using Spherical Designs , 2013, Journal of Fourier Analysis and Applications.

[41]  Babak Hassibi,et al.  Sparse phase retrieval: Convex algorithms and limitations , 2013, 2013 IEEE International Symposium on Information Theory.

[42]  J. Miao,et al.  Erratum: High Resolution 3D X-Ray Diffraction Microscopy [Phys. Rev. Lett.89, 088303 (2002)] , 2013 .

[43]  Anima Anandkumar,et al.  Exact Recovery of Sparsely Used Overcomplete Dictionaries , 2013, ArXiv.

[44]  Yoram Bresler,et al.  Near Optimal Compressed Sensing of Sparse Rank-One Matrices via Sparse Power Factorization , 2013, ArXiv.

[45]  Tetsunao Matsuta,et al.  国際会議開催報告:2013 IEEE International Symposium on Information Theory , 2013 .

[46]  Xiaodong Li,et al.  Sparse Signal Recovery from Quadratic Measurements via Convex Programming , 2012, SIAM J. Math. Anal..

[47]  T. Heinosaari,et al.  Quantum Tomography under Prior Information , 2011, 1109.5478.

[48]  Prateek Jain,et al.  Low-rank matrix completion using alternating minimization , 2012, STOC '13.

[49]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[50]  Michel Verhaegen,et al.  Quadratic Basis Pursuit , 2013, 1301.7002.

[51]  Anima Anandkumar,et al.  Guaranteed Non-Orthogonal Tensor Decomposition via Alternating Rank-1 Updates , 2014, ArXiv.

[52]  Bo Huang,et al.  Square Deal: Lower Bounds and Improved Relaxations for Tensor Recovery , 2013, ICML.

[53]  A. Appendix.  Alternating Minimization for Mixed Linear Regression , 2014 .

[54]  Bamdev Mishra,et al.  Manopt, a matlab toolbox for optimization on manifolds , 2013, J. Mach. Learn. Res..

[55]  Prateek Jain,et al.  Provable Tensor Factorization with Missing Data , 2014, NIPS.

[56]  Anima Anandkumar,et al.  Analyzing Tensor Power Method Dynamics: Applications to Learning Overcomplete Latent Variable Models , 2014, ArXiv.

[57]  Zhiqiang Xu,et al.  A strong restricted isometry property, with an application to phaseless compressed sensing , 2014, ArXiv.

[58]  Constantine Caramanis,et al.  Alternating Minimization for Mixed Linear Regression , 2013, ICML.

[59]  Prateek Jain,et al.  Non-convex Robust PCA , 2014, NIPS.

[60]  Xiaodong Li,et al.  Solving Quadratic Equations via PhaseLift When There Are About as Many Equations as Unknowns , 2012, Found. Comput. Math..

[61]  Mary Wootters,et al.  Fast matrix completion without the condition number , 2014, COLT.

[62]  Dustin G. Mixon,et al.  Phase Retrieval with Polarization , 2012, SIAM J. Imaging Sci..

[63]  Aditya Bhaskara,et al.  More Algorithms for Provable Dictionary Learning , 2014, ArXiv.

[64]  Anima Anandkumar,et al.  Provable Tensor Methods for Learning Mixtures of Classifiers , 2014, ArXiv.

[65]  Sanjeev Arora,et al.  New Algorithms for Learning Incoherent and Overcomplete Dictionaries , 2013, COLT.

[66]  Moritz Hardt,et al.  Understanding Alternating Minimization for Matrix Completion , 2013, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[67]  Yonina C. Eldar,et al.  GESPAR: Efficient Phase Retrieval of Sparse Signals , 2013, IEEE Transactions on Signal Processing.

[68]  Sujay Sanghavi,et al.  The Local Convexity of Solving Quadratic Equations , 2015 .

[69]  John Wright,et al.  Square Deal : Lower Bounds and Improved Convex Relaxations for Tensor Recovery , 2015 .

[70]  John Wright,et al.  Complete dictionary recovery over the sphere , 2015, 2015 International Conference on Sampling Theory and Applications (SampTA).

[71]  Prateek Jain,et al.  Computing Matrix Squareroot via Non Convex Local Search , 2015, ArXiv.

[72]  Prateek Jain,et al.  Fast Exact Matrix Completion with Finite Samples , 2014, COLT.

[73]  Michael B. Wakin,et al.  Greed is Super: A Fast Algorithm for Super-Resolution , 2015, ArXiv.

[74]  Kiryung Lee,et al.  RIP-like Properties in Subsampled Blind Deconvolution , 2015, ArXiv.

[75]  Yonina C. Eldar,et al.  Phase Retrieval: An Overview of Recent Developments , 2015, ArXiv.

[76]  Alexandre d'Aspremont,et al.  Phase recovery, MaxCut and complex semidefinite programming , 2012, Math. Program..

[77]  Sanjeev Arora,et al.  Simple, Efficient, and Neural Algorithms for Sparse Coding , 2015, COLT.

[78]  Xiaodong Li,et al.  Optimal Rates of Convergence for Noisy Sparse Phase Retrieval via Thresholded Wirtinger Flow , 2015, ArXiv.

[79]  Yuxin Chen,et al.  Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems , 2015, NIPS.

[80]  Zhi-Quan Luo,et al.  Guaranteed Matrix Completion via Non-Convex Factorization , 2014, IEEE Transactions on Information Theory.

[81]  Prateek Jain,et al.  Phase Retrieval Using Alternating Minimization , 2013, IEEE Transactions on Signal Processing.

[82]  John D. Lafferty,et al.  A Convergent Gradient Descent Algorithm for Rank Minimization and Semidefinite Programming from Random Linear Measurements , 2015, NIPS.

[83]  Yonina C. Eldar,et al.  Phase Retrieval with Application to Optical Imaging: A contemporary overview , 2015, IEEE Signal Processing Magazine.

[84]  Furong Huang,et al.  Escaping From Saddle Points - Online Stochastic Gradient for Tensor Decomposition , 2015, COLT.

[85]  Yonina C. Eldar,et al.  Simultaneously Structured Models With Application to Sparse and Low-Rank Matrices , 2012, IEEE Transactions on Information Theory.

[86]  Christopher De Sa,et al.  Global Convergence of Stochastic Gradient Descent for Some Non-convex Matrix Problems , 2014, ICML.

[87]  Tselil Schramm,et al.  Speeding up sum-of-squares for tensor decomposition and planted sparse vectors , 2015, ArXiv.

[88]  Zhang Fe.  Phase retrieval from coded diffraction patterns , 2015 .

[89]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[90]  John Wright,et al.  When Are Nonconvex Problems Not Scary? , 2015, ArXiv.

[91]  Yonina C. Eldar,et al.  Phase Retrieval via Matrix Completion , 2011, SIAM Rev..

[92]  Martin J. Wainwright,et al.  Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees , 2015, ArXiv.

[93]  Kenji Kawaguchi,et al.  Deep Learning without Poor Local Minima , 2016, NIPS.

[94]  Vladislav Voroninski,et al.  Compressed Sensing from Phaseless Gaussian Measurements via Linear Programming in the Natural Parameter Space , 2016, ArXiv.

[95]  Yingbin Liang,et al.  Provable Non-convex Phase Retrieval with Outliers: Median TruncatedWirtinger Flow , 2016, ICML.

[96]  Nicolas Boumal,et al.  The non-convex Burer-Monteiro approach works on smooth semidefinite programs , 2016, NIPS.

[97]  Ayfer Özgür,et al.  Phase Retrieval via Incremental Truncated Wirtinger Flow , 2016, ArXiv.

[98]  Ju Sun,et al.  When Are Nonconvex Optimization Problems Not Scary? , 2016 .

[99]  John Wright,et al.  A Geometric Analysis of Phase Retrieval , 2016, International Symposium on Information Theory.

[100]  Yingbin Liang,et al.  Reshaped Wirtinger Flow for Solving Quadratic System of Equations , 2016, NIPS.

[101]  Nathan Srebro,et al.  Global Optimality of Local Search for Low Rank Matrix Recovery , 2016, NIPS.

[102]  John Wright,et al.  Finding a Sparse Vector in a Subspace: Linear Sparsity Using Alternating Directions , 2014, IEEE Transactions on Information Theory.

[103]  Vladislav Voroninski,et al.  An Elementary Proof of Convex Phase Retrieval in the Natural Parameter Space via the Linear Program PhaseMax , 2016, ArXiv.

[104]  Nicolas Boumal,et al.  On the low-rank approach for semidefinite programs arising in synchronization and community detection , 2016, COLT.

[105]  Prateek Jain,et al.  Tensor vs. Matrix Methods: Robust Tensor Decomposition under Block Sparse Perturbations , 2015, AISTATS.

[106]  Anima Anandkumar,et al.  Efficient approaches for escaping higher order saddle points in non-convex optimization , 2016, COLT.

[107]  Bing Gao,et al.  Gauss-Newton Method for Phase Retrieval , 2016, ArXiv.

[108]  Daniel Soudry,et al.  No bad local minima: Data independent training error guarantees for multilayer neural networks , 2016, ArXiv.

[109]  Michael I. Jordan,et al.  Gradient Descent Converges to Minimizers , 2016, ArXiv.

[110]  Tengyu Ma,et al.  Matrix Completion has No Spurious Local Minimum , 2016, NIPS.

[111]  Prateek Jain,et al.  Learning Sparsely Used Overcomplete Dictionaries via Alternating Minimization , 2013, SIAM J. Optim..

[112]  Max Simchowitz,et al.  Low-rank Solutions of Linear Matrix Equations via Procrustes Flow , 2015, ICML.

[113]  Tony F. Chan,et al.  Guarantees of Riemannian Optimization for Low Rank Matrix Recovery , 2015, SIAM J. Matrix Anal. Appl..

[114]  Nicolas Boumal,et al.  Nonconvex Phase Synchronization , 2016, SIAM J. Optim..

[115]  D. Gleich.  TRUST REGION METHODS , 2017 .

[116]  Justin Romberg,et al.  Phase Retrieval Meets Statistical Learning Theory: A Flexible Convex Relaxation , 2016, AISTATS.

[117]  Georgios Piliouras,et al.  Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions , 2016, ITCS.

[118]  Gang Wang,et al.  Solving large-scale systems of random quadratic equations via stochastic truncated amplitude flow , 2017, 2017 25th European Signal Processing Conference (EUSIPCO).

[119]  Yanjun Li,et al.  Blind Recovery of Sparse Signals From Subsampled Convolution , 2015, IEEE Transactions on Information Theory.

[120]  Bing Gao,et al.  Phaseless Recovery Using the Gauss–Newton Method , 2016, IEEE Transactions on Signal Processing.

[121]  Anastasios Kyrillidis,et al.  Non-square matrix sensing without spurious local minima via the Burer-Monteiro approach , 2016, AISTATS.

[122]  Irène Waldspurger,et al.  Phase Retrieval With Random Gaussian Sensing Vectors by Alternating Projections , 2016, IEEE Transactions on Information Theory.

[123]  Yonina C. Eldar,et al.  Non-Convex Phase Retrieval From STFT Measurements , 2016, IEEE Transactions on Information Theory.

[124]  Yonina C. Eldar,et al.  Solving Systems of Random Quadratic Equations via Truncated Amplitude Flow , 2016, IEEE Transactions on Information Theory.

[125]  Tom Goldstein,et al.  PhaseMax: Convex Phase Retrieval via Basis Pursuit , 2016, IEEE Transactions on Information Theory.

[126]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .