Median-Truncated Nonconvex Approach for Phase Retrieval With Outliers

This paper investigates the phase retrieval problem, which aims to recover a signal from the magnitudes of its linear measurements. We develop statistically and computationally efficient algorithms for the situation when the measurements are corrupted by sparse outliers that can take arbitrary values. We propose a novel approach to robustify the gradient descent algorithm by using the sample median as a guide for pruning spurious samples in initialization and local search. Adopting a Poisson loss and a reshaped quadratic loss, respectively, we obtain two algorithms termed median-truncated Wirtinger flow and median-reshaped Wirtinger flow, both of which provably recover the signal from a near-optimal number of measurements when the measurement vectors are composed of independent and identically distributed Gaussian entries, up to a logarithmic factor, even when a constant fraction of the measurements is adversarially corrupted. We further show that both algorithms are stable in the presence of additional dense bounded noise. Our analysis is accomplished by developing non-trivial concentration results of median-related quantities, which may be of independent interest. We provide numerical experiments to demonstrate the effectiveness of our approach.

[1]  John Wright,et al.  A Geometric Analysis of Phase Retrieval , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[2]  Irène Waldspurger,et al.  Phase Retrieval With Random Gaussian Sensing Vectors by Alternating Projections , 2016, IEEE Transactions on Information Theory.

[3]  Nathan Srebro,et al.  Global Optimality of Local Search for Low Rank Matrix Recovery , 2016, NIPS.

[4]  Alexandre d'Aspremont,et al.  Phase recovery, MaxCut and complex semidefinite programming , 2012, Math. Program..

[5]  Huan Xu,et al.  Subspace Clustering with Irrelevant Features via Robust Dantzig Selector , 2015, NIPS.

[6]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[7]  M. Smyth,et al.  x Ray crystallography , 2000, Methods of biochemical analysis.

[8]  Prateek Jain,et al.  Low-rank matrix completion using alternating minimization , 2012, STOC '13.

[9]  Junwei Lu,et al.  Symmetry, Saddle Points, and Global Geometry of Nonconvex Matrix Factorization , 2016, ArXiv.

[10]  Moses Charikar,et al.  Finding frequent items in data streams , 2002, Theor. Comput. Sci..

[11]  Prateek Jain,et al.  Phase Retrieval Using Alternating Minimization , 2013, IEEE Transactions on Signal Processing.

[12]  Yonina C. Eldar,et al.  Solving Systems of Random Quadratic Equations via Truncated Amplitude Flow , 2016, IEEE Transactions on Information Theory.

[13]  Andrea J. Goldsmith,et al.  Exact and Stable Covariance Estimation From Quadratic Sampling via Convex Programming , 2013, IEEE Transactions on Information Theory.

[14]  Anastasios Kyrillidis,et al.  Provable non-convex projected gradient descent for a class of constrained matrix optimization problems , 2016, ArXiv.

[15]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[16]  Yuejie Chi,et al.  Reshaped Wirtinger Flow and Incremental Algorithm for Solving Quadratic System of Equations , 2016 .

[17]  Justin Romberg,et al.  Phase Retrieval Meets Statistical Learning Theory: A Flexible Convex Relaxation , 2016, AISTATS.

[18]  Junwei Lu,et al.  Symmetry. Saddle Points, and Global Optimization Landscape of Nonconvex Matrix Factorization , 2016, 2018 Information Theory and Applications Workshop (ITA).

[19]  Yuxin Chen,et al.  Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems , 2015, NIPS.

[20]  Yingbin Liang,et al.  Nonconvex Low-Rank Matrix Recovery with Arbitrary Outliers via Median-Truncated Gradient Descent , 2017, Information and Inference: A Journal of the IMA.

[21]  Max Simchowitz,et al.  Low-rank Solutions of Linear Matrix Equations via Procrustes Flow , 2015, ICML.

[22]  Yudong Chen,et al.  Harnessing Structures in Big Data via Guaranteed Low-Rank Matrix Estimation: Recent Theory and Fast Algorithms via Convex and Nonconvex Optimization , 2018, IEEE Signal Processing Magazine.

[23]  John Stillwell,et al.  Symmetry , 2000, Am. Math. Mon..

[24]  Emmanuel J. Candès,et al.  PhaseLift: Exact and Stable Signal Recovery from Magnitude Measurements via Convex Programming , 2011, ArXiv.

[25]  R. Balan,et al.  On signal reconstruction without phase , 2006 .

[26]  John D. Lafferty,et al.  Convergence Analysis for Rectangular Matrix Completion Using Burer-Monteiro Factorization and Gradient Descent , 2016, ArXiv.

[27]  A. M. B. DOUGLAS,et al.  X-Ray Crystallography , 1947, Nature.

[28]  Nicolas Boumal,et al.  Nonconvex Phase Synchronization , 2016, SIAM J. Optim..

[29]  A. Föhrenbach,et al.  SIMPLE++ , 2000, OR Spectr..

[30]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[31]  Yonina C. Eldar,et al.  Convolutional Phase Retrieval , 2017, NIPS.

[32]  Christopher De Sa,et al.  Global Convergence of Stochastic Gradient Descent for Some Non-convex Matrix Problems , 2014, ICML.

[33]  Xiaodong Li,et al.  Rapid, Robust, and Reliable Blind Deconvolution via Nonconvex Optimization , 2016, Applied and Computational Harmonic Analysis.

[34]  Yuanying Chen,et al.  Rapid evolution of piRNA clusters in the Drosophila melanogaster ovary , 2023, bioRxiv.

[35]  Anastasios Kyrillidis,et al.  Provable Burer-Monteiro factorization for a class of norm-constrained matrix problems , 2016 .

[36]  Tengyu Ma,et al.  Matrix Completion has No Spurious Local Minimum , 2016, NIPS.

[37]  J. D. Donahue PRODUCTS AND QUOTIENTS OF RANDOM VARIABLES AND THEIR APPLICATIONS , 1964 .

[38]  Xiaodong Li,et al.  Solving Quadratic Equations via PhaseLift When There Are About as Many Equations as Unknowns , 2012, Found. Comput. Math..

[39]  Sujay Sanghavi,et al.  The Local Convexity of Solving Systems of Quadratic Equations , 2015, 1506.07868.

[40]  Yuxin Chen,et al.  The Projected Power Method: An Efficient Algorithm for Joint Alignment from Pairwise Differences , 2016, Communications on Pure and Applied Mathematics.

[41]  Ryan J. Tibshirani,et al.  Fast computation of the median by successive binning , 2008, ArXiv.

[42]  Constantine Caramanis,et al.  A Convex Formulation for Mixed Regression: Near Optimal Rates in the Face of Noise , 2013, ArXiv.

[43]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, 2009 IEEE International Symposium on Information Theory.

[44]  Yingbin Liang,et al.  A Nonconvex Approach for Phase Retrieval: Reshaped Wirtinger Flow and Incremental Algorithms , 2017, J. Mach. Learn. Res..

[45]  Ke Chen,et al.  On k-Median clustering in high dimensions , 2006, SODA '06.

[46]  Paul Hand,et al.  PhaseLift is robust to a constant fraction of arbitrary errors , 2015, 1502.04241.

[47]  Shie Mannor,et al.  Robust Sparse Regression under Adversarial Corruption , 2013, ICML.

[48]  Moritz Hardt,et al.  Understanding Alternating Minimization for Matrix Completion , 2013, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[49]  Tom Goldstein,et al.  PhaseMax: Convex Phase Retrieval via Basis Pursuit , 2016, IEEE Transactions on Information Theory.

[50]  John D. Lafferty,et al.  A Convergent Gradient Descent Algorithm for Rank Minimization and Semidefinite Programming from Random Linear Measurements , 2015, NIPS.

[51]  Vladislav Voroninski,et al.  An Elementary Proof of Convex Phase Retrieval in the Natural Parameter Space via the Linear Program PhaseMax , 2016, ArXiv.

[52]  Xiaodong Li,et al.  Optimal Rates of Convergence for Noisy Sparse Phase Retrieval via Thresholded Wirtinger Flow , 2015, ArXiv.

[53]  Constantine Caramanis,et al.  Fast Algorithms for Robust PCA via Gradient Descent , 2016, NIPS.

[54]  Jeffrey A. Fessler,et al.  Undersampled Phase Retrieval With Outliers , 2014, IEEE Transactions on Computational Imaging.

[55]  Tony F. Chan,et al.  Guarantees of Riemannian Optimization for Low Rank Matrix Recovery , 2015, SIAM J. Matrix Anal. Appl..

[56]  Xiaodong Li,et al.  Sparse Signal Recovery from Quadratic Measurements via Convex Programming , 2012, SIAM J. Math. Anal..

[57]  Yue Sun,et al.  Low-Rank Positive Semidefinite Matrix Recovery From Corrupted Rank-One Measurements , 2016, IEEE Transactions on Signal Processing.

[58]  Sham M. Kakade,et al.  Provable Efficient Online Matrix Completion via Non-convex Stochastic Gradient Descent , 2016, NIPS.

[59]  Yuxin Chen,et al.  Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution , 2017, Found. Comput. Math..

[60]  Xiaodong Li,et al.  Phase Retrieval from Coded Diffraction Patterns , 2013, 1310.3240.

[61]  Martin J. Wainwright,et al.  Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees , 2015, ArXiv.

[62]  Nicolas Boumal,et al.  On the low-rank approach for semidefinite programs arising in synchronization and community detection , 2016, COLT.

[63]  Xiao Zhang,et al.  Robust Wirtinger Flow for Phase Retrieval with Arbitrary Corruption , 2017, ArXiv.

[64]  Sanjeev Arora,et al.  Simple, Efficient, and Neural Algorithms for Sparse Coding , 2015, COLT.

[65]  John Wright,et al.  Complete Dictionary Recovery Using Nonconvex Optimization , 2015, ICML.

[66]  Prateek Jain,et al.  Non-convex Robust PCA , 2014, NIPS.

[67]  Yuxin Chen,et al.  Gradient descent with random initialization: fast global convergence for nonconvex phase retrieval , 2018, Mathematical Programming.

[68]  Vladislav Voroninski,et al.  Corruption Robust Phase Retrieval via Linear Programming , 2016, ArXiv.

[69]  David A. Wagner,et al.  Resilient aggregation in sensor networks , 2004, SASN '04.

[70]  Prateek Jain,et al.  Tensor vs. Matrix Methods: Robust Tensor Decomposition under Block Sparse Perturbations , 2015, AISTATS.

[71]  L. Demanet,et al.  Stable Optimizationless Recovery from Phaseless Linear Measurements , 2012, Journal of Fourier Analysis and Applications.

[72]  Zhi-Quan Luo,et al.  Guaranteed Matrix Completion via Non-Convex Factorization , 2014, IEEE Transactions on Information Theory.

[73]  Nikos D. Sidiropoulos,et al.  Inexact Alternating Optimization for Phase Retrieval in the Presence of Outliers , 2016, IEEE Transactions on Signal Processing.

[74]  Gongguo Tang,et al.  The nonconvex geometry of low-rank matrix optimizations with general objective functions , 2016, 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[75]  Yanjun Li,et al.  Blind Recovery of Sparse Signals From Subsampled Convolution , 2015, IEEE Transactions on Information Theory.

[76]  Kubilay Sertel,et al.  Phase-sensitive THz imaging using single-pixel intensity-only measurements , 2016, 2016 IEEE International Symposium on Antennas and Propagation (APSURSI).

[77]  J R Fienup,et al.  Phase retrieval algorithms: a comparison. , 1982, Applied optics.