Local saddles of relaxed averaged alternating reflections algorithms on phase retrieval

Phase retrieval can be expressed as a non-convex constrained optimization problem to identify one phase minimizer one a torus. Many iterative transform techniques have been proposed to identify the minimizer, e.g., relaxed averaged alternating reflections (RAAR) algorithms. In this paper, we present one optimization viewpoint on the RAAR algorithm. RAAR algorithm is one alternating direction method of multipliers with one penalty parameter. Pairing with multipliers (dual vectors), phase vectors on the primal space are lifted to higher dimensional vectors, RAAR algorithm is one continuation algorithm, which searches for local saddles in the primal-dual space. The dual iteration approximates one gradient ascent flow, which drives the corresponding local minimizers in a positive-definite Hessian region. Altering penalty parameters, the RAAR avoids the stagnation of these corresponding local minimizers in the primal space and thus screens out many stationary points corresponding to non-local minimizers.

[1]  Tie Zhou,et al.  On relaxed averaged alternating reflections (RAAR) algorithm for phase retrieval with structured illumination , 2016, 1607.02138.

[2]  Yonina C. Eldar,et al.  Phase Retrieval: Stability and Recovery Guarantees , 2012, ArXiv.

[3]  Prateek Jain,et al.  Phase Retrieval Using Alternating Minimization , 2013, IEEE Transactions on Signal Processing.

[4]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[5]  Junfeng Yang,et al.  An Efficient TVL1 Algorithm for Deblurring Multichannel Images Corrupted by Impulsive Noise , 2009, SIAM J. Sci. Comput..

[6]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[7]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[8]  Yue M. Lu,et al.  Phase Transitions of Spectral Initialization for High-Dimensional Nonconvex Estimation , 2017, Information and Inference: A Journal of the IMA.

[9]  Albert Fannjiang,et al.  Absolute uniqueness of phase retrieval with random illumination , 2011, ArXiv.

[10]  Pengwen Chen,et al.  Phase Retrieval by Linear Algebra , 2017, SIAM J. Matrix Anal. Appl..

[11]  Michael I. Jordan,et al.  Gradient Descent Only Converges to Minimizers , 2016, COLT.

[12]  Guoyin Li,et al.  Douglas–Rachford splitting for nonconvex optimization with application to nonconvex feasibility problems , 2014, Math. Program..

[13]  Bingsheng He,et al.  On the convergence rate of Douglas–Rachford operator splitting method , 2015, Math. Program..

[14]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[15]  Michael I. Jordan,et al.  On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems , 2019, ICML.

[16]  Yue M. Lu,et al.  Optimal Spectral Initialization for Signal Recovery With Applications to Phase Retrieval , 2018, IEEE Transactions on Signal Processing.

[17]  ASHISH CHERUKURI,et al.  Saddle-Point Dynamics: Conditions for Asymptotic Stability of Saddle Points , 2015, SIAM J. Control. Optim..

[18]  R. Gerchberg.  A practical algorithm for the determination of phase from image and diffraction plane pictures , 1972 .

[19]  Pablo Enfedaque,et al.  Blind Ptychographic Phase Retrieval via Convergent Alternating Direction Method of Multipliers , 2018, SIAM J. Imaging Sci..

[20]  Michael I. Jordan,et al.  How to Escape Saddle Points Efficiently , 2017, ICML.

[21]  Feng Ruan,et al.  Solving (most) of a set of quadratic equalities: Composite optimization for robust phase retrieval , 2017, Information and Inference: A Journal of the IMA.

[22]  Heinz H. Bauschke,et al.  Hybrid projection-reflection method for phase retrieval. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[23]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[24]  GoldfarbDonald,et al.  Bregman Iterative Algorithms for $\ell_1$-Minimization with Applications to Compressed Sensing , 2008 .

[25]  Michael I. Jordan,et al.  Gradient Descent Can Take Exponential Time to Escape Saddle Points , 2017, NIPS.

[26]  Pengwen Chen,et al.  Fourier Phase Retrieval with a Single Mask by Douglas-Rachford Algorithm , 2015, Applied and computational harmonic analysis.

[27]  Wotao Yin,et al.  Global Convergence of ADMM in Nonconvex Nonsmooth Optimization , 2015, Journal of Scientific Computing.

[28]  Chao Yang,et al.  Alternating direction methods for classical and ptychographic phase retrieval , 2012 .

[29]  Michael I. Jordan,et al.  What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization? , 2019, ICML.

[30]  Yonina C. Eldar,et al.  Phase Retrieval with Application to Optical Imaging: A contemporary overview , 2015, IEEE Signal Processing Magazine.

[31]  Chi Jin.  What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization? , 2020 .

[32]  Liwei Zhang,et al.  Optimality Conditions for Constrained Minimax Optimization , 2020, CSIAM Transactions on Applied Mathematics.

[33]  John Wright,et al.  A Geometric Analysis of Phase Retrieval , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[34]  Jonathan P. How,et al.  Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability , 2017, ICML.

[35]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[36]  Guoyin Li,et al.  Global Convergence of Splitting Methods for Nonconvex Composite Optimization , 2014, SIAM J. Optim..

[37]  Michael K. Ng,et al.  Phase Retrieval from Incomplete Magnitude Information via Total Variation Regularization , 2016, SIAM J. Sci. Comput..

[38]  Bhabesh Deka,et al.  Compressed Sensing Magnetic Resonance Image Reconstruction Algorithms , 2018, Springer Series on Bio- and Neurosystems.

[39]  D. R. Luke.  Relaxed averaged alternating reflections for diffraction imaging , 2004, math/0405208.

[40]  Yuxin Chen,et al.  Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems , 2015, NIPS.

[41]  A. Fannjiang,et al.  Phase Retrieval with One or Two Diffraction Patterns by Alternating Projections with the Null Initialization , 2015, 1510.07379.

[42]  A. Fannjiang,et al.  Fixed Point Analysis of Douglas-Rachford Splitting for Ptychography and Phase Retrieval , 2020, SIAM J. Imaging Sci..

[43]  Zhi-Quan Luo,et al.  Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems , 2014, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[44]  P. Thibault,et al.  Maximum-likelihood refinement for coherent diffractive imaging , 2012 .

[45]  Andrea Montanari,et al.  Fundamental Limits of Weak Recovery with Applications to Phase Retrieval , 2017, COLT.

[46]  Constantinos Daskalakis,et al.  The Limit Points of (Optimistic) Gradient Descent in Min-Max Optimization , 2018, NeurIPS.

[47]  James R Fienup,et al.  Phase retrieval algorithms: a personal tour [Invited]. , 2013, Applied optics.

[48]  Thomas Hofmann,et al.  Local Saddle Point Optimization: A Curvature Exploitation Approach , 2018, AISTATS.

[49]  J R Fienup,et al.  Phase retrieval algorithms: a comparison. , 1982, Applied optics.

[50]  Yingbin Liang,et al.  Reshaped Wirtinger Flow for Solving Quadratic System of Equations , 2016, NIPS.