Optimization-Based AMP for Phase Retrieval: The Impact of Initialization and $\ell_{2}$ Regularization

We consider an <inline-formula> <tex-math notation="LaTeX">$\ell _{2}$ </tex-math></inline-formula>-regularized non-convex optimization problem for recovering signals from their noisy phaseless observations. We design and study the performance of a message passing algorithm that aims to solve this optimization problem. We consider the asymptotic setting <inline-formula> <tex-math notation="LaTeX">$m,n \rightarrow \infty $ </tex-math></inline-formula>, <inline-formula> <tex-math notation="LaTeX">$m/n \rightarrow \delta $ </tex-math></inline-formula> and obtain sharp performance bounds, where <inline-formula> <tex-math notation="LaTeX">$m$ </tex-math></inline-formula> is the number of measurements and <inline-formula> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> is the signal dimension. We show that for complex signals, the algorithm can perform accurate recovery with only <inline-formula> <tex-math notation="LaTeX">$m = (({64}/{\pi ^{2}})-4)n \approx 2.5n$ </tex-math></inline-formula> measurements. Also, we provide a sharp analysis on the sensitivity of the algorithm to noise. We highlight the following facts about our message passing algorithm: 1) adding <inline-formula> <tex-math notation="LaTeX">$\ell _{2}$ </tex-math></inline-formula> regularization to the non-convex loss function can be beneficial and 2) spectral initialization has a marginal impact on the performance of the algorithm. The sharp analyses, in this paper, not only enable us to compare the performance of our method with other phase recovery schemes but also shed light on designing better iterative algorithms for other non-convex optimization problems.

[1]  Gang Wang,et al.  Solving Most Systems of Random Quadratic Equations , 2017, NIPS.

[2]  Milad Bakhshizadeh,et al.  Compressive Phase Retrieval of Structured Signal , 2017, ArXiv.

[3]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[4]  Dustin G. Mixon,et al.  Saving phase: Injectivity and stability for phase retrieval , 2013, 1302.4618.

[5]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[6]  Richard G. Baraniuk,et al.  Consistent Parameter Estimation for LASSO and Approximate Message Passing , 2015, The Annals of Statistics.

[7]  Yuxin Chen,et al.  Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems , 2015, NIPS.

[8]  Babak Hassibi,et al.  Performance of real phase retrieval , 2017, 2017 International Conference on Sampling Theory and Applications (SampTA).

[9]  Rick P. Millane,et al.  Phase retrieval in crystallography and optics , 1990 .

[10]  Ling Zhu,et al.  ON A QUADRATIC ESTIMATE OF SHAFER , 2008 .

[11]  Tom Goldstein,et al.  PhaseMax: Convex Phase Retrieval via Basis Pursuit , 2016, IEEE Transactions on Information Theory.

[12]  Milad Bakhshizadeh,et al.  Using Black-Box Compression Algorithms for Phase Retrieval , 2017, IEEE Transactions on Information Theory.

[13]  Xiaodong Li,et al.  Solving Quadratic Equations via PhaseLift When There Are About as Many Equations as Unknowns , 2012, Found. Comput. Math..

[14]  Yingbin Liang,et al.  Reshaped Wirtinger Flow for Solving Quadratic System of Equations , 2016, NIPS.

[15]  Emmanuel J. Candès,et al.  PhaseLift: Exact and Stable Signal Recovery from Magnitude Measurements via Convex Programming , 2011, ArXiv.

[16]  Nicolas Macris,et al.  Phase Transitions, Optimal Errors and Optimality of Message-Passing in Generalized Linear Models , 2017, ArXiv.

[17]  Christos Thrampoulidis,et al.  Precise Error Analysis of Regularized $M$ -Estimators in High Dimensions , 2016, IEEE Transactions on Information Theory.

[18]  Christos Thrampoulidis,et al.  Regularized Linear Regression: A Precise Analysis of the Estimation Error , 2015, COLT.

[19]  R. Balan,et al.  On signal reconstruction without phase , 2006 .

[20]  Andrea Montanari,et al.  Fundamental Limits of Weak Recovery with Applications to Phase Retrieval , 2017, COLT.

[21]  Richard G. Baraniuk,et al.  Asymptotic Analysis of Complex LASSO via Complex Approximate Message Passing (CAMP) , 2011, IEEE Transactions on Information Theory.

[22]  Yuxin Chen,et al.  Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution , 2017, Found. Comput. Math..

[23]  Hossein Hassani,et al.  On the Folded Normal Distribution , 2014, 1402.3559.

[24]  Paul F. Byrd,et al.  Handbook of elliptic integrals for engineers and scientists , 1971 .

[25]  Damek Davis,et al.  The nonsmooth landscape of phase retrieval , 2017, IMA Journal of Numerical Analysis.

[26]  Radu Balan,et al.  Reconstruction of Signals from Magnitudes of Redundant Representations: The Complex Case , 2012, Found. Comput. Math..

[27]  Yu-Ming Chu,et al.  Asymptotical Bounds for Complete Elliptic Integrals of the Second Kind , 2012, 1209.0066.

[28]  Simon Campese,et al.  Fourth Moment Theorems for complex Gaussian approximation , 2015, 1511.00547.

[29]  Yonina C. Eldar,et al.  Convolutional Phase Retrieval via Gradient Descent , 2017, IEEE Transactions on Information Theory.

[30]  Adel Javanmard,et al.  State Evolution for General Approximate Message Passing Algorithms, with Applications to Spatial Coupling , 2012, ArXiv.

[31]  Yan Shuo Tan,et al.  Phase Retrieval via Randomized Kaczmarz: Theoretical Guarantees , 2017, ArXiv.

[32]  A. Maleki,et al.  From compression to compressed sensing , 2016 .

[33]  Yuantao Gu,et al.  Phase retrieval using iterative projections: Dynamics in the large systems limit , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[34]  Yue M. Lu,et al.  Phase Transitions of Spectral Initialization for High-Dimensional Nonconvex Estimation , 2017, Information and Inference: A Journal of the IMA.

[35]  Sundeep Rangan,et al.  Compressive phase retrieval via generalized approximate message passing , 2012, Allerton Conference.

[36]  Alexandre d'Aspremont,et al.  Phase recovery, MaxCut and complex semidefinite programming , 2012, Math. Program..

[37]  Mahdi Soltanolkotabi,et al.  Structured Signal Recovery From Quadratic Measurements: Breaking Sample Complexity Barriers via Nonconvex Optimization , 2017, IEEE Transactions on Information Theory.

[38]  Yue M. Lu,et al.  Fundamental limits of phasemax for phase retrieval: A replica analysis , 2017, 2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[39]  Xiaodong Li,et al.  Optimal Rates of Convergence for Noisy Sparse Phase Retrieval via Thresholded Wirtinger Flow , 2015, ArXiv.

[40]  John Wright,et al.  A Geometric Analysis of Phase Retrieval , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[41]  Yonina C. Eldar,et al.  Solving Systems of Random Quadratic Equations via Truncated Amplitude Flow , 2016, IEEE Transactions on Information Theory.

[42]  Sundeep Rangan,et al.  Generalized approximate message passing for estimation with random linear mixing , 2010, 2011 IEEE International Symposium on Information Theory Proceedings.

[43]  Babak Hassibi,et al.  A Precise Analysis of PhaseMax in Phase Retrieval , 2018, 2018 IEEE International Symposium on Information Theory (ISIT).

[44]  Andrea Montanari,et al.  The dynamics of message passing on dense graphs, with applications to compressed sensing , 2010, 2010 IEEE International Symposium on Information Theory.

[45]  C. Sinan Güntürk,et al.  Convergence of the randomized Kaczmarz method for phase retrieval , 2017, ArXiv.

[46]  Feng Ruan,et al.  Solving (most) of a set of quadratic equalities: Composite optimization for robust phase retrieval , 2017, Information and Inference: A Journal of the IMA.

[47]  Yonina C. Eldar,et al.  Phase Retrieval: Stability and Recovery Guarantees , 2012, ArXiv.

[48]  Ke Wei Solving systems of phaseless equations via Kaczmarz methods: a proof of concept study , 2015 .

[49]  Andrea Montanari,et al.  The LASSO Risk for Gaussian Matrices , 2010, IEEE Transactions on Information Theory.

[50]  Prateek Jain,et al.  Phase Retrieval Using Alternating Minimization , 2013, IEEE Transactions on Signal Processing.

[51]  Justin Romberg,et al.  Phase Retrieval Meets Statistical Learning Theory: A Flexible Convex Relaxation , 2016, AISTATS.

[52]  Hing-Cheung So,et al.  Coordinate Descent Algorithms for Phase Retrieval , 2017, Signal Process..

[53]  Yin Zhang,et al.  Fixed-Point Continuation for l1-Minimization: Methodology and Convergence , 2008, SIAM J. Optim..

[54]  Yuejie Chi,et al.  Kaczmarz Method for Solving Quadratic Equations , 2016, IEEE Signal Processing Letters.

[55]  Christos Thrampoulidis,et al.  Phase retrieval via linear programming: Fundamental limits and algorithmic improvements , 2017, 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton).