Solving large-scale systems of random quadratic equations via stochastic truncated amplitude flow

A novel approach termed <italic>stochastic truncated amplitude flow</italic> (STAF) is developed to reconstruct an unknown <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula>-dimensional real-/complex-valued signal <inline-formula><tex-math notation="LaTeX">$\boldsymbol {x}$</tex-math></inline-formula> from <inline-formula> <tex-math notation="LaTeX">$m$</tex-math></inline-formula> “phaseless” quadratic equations of the form <inline-formula><tex-math notation="LaTeX">$\psi _i=|\langle \boldsymbol {a}_i,\boldsymbol {x}\rangle |$</tex-math> </inline-formula>. This problem, also known as phase retrieval from magnitude-only information, is <italic>NP-hard </italic> in general. Adopting an amplitude-based nonconvex formulation, STAF leads to an iterative solver comprising two stages: s1) Orthogonality-promoting initialization through a stochastic variance reduced gradient algorithm; and, s2) a series of iterative refinements of the initialization using stochastic truncated gradient iterations. Both stages involve a single equation per iteration, thus rendering STAF a simple, scalable, and fast approach amenable to large-scale implementations that are useful when <inline-formula><tex-math notation="LaTeX">$n$</tex-math> </inline-formula> is large. When <inline-formula><tex-math notation="LaTeX">$\lbrace \boldsymbol {a}_i\rbrace _{i=1}^m$ </tex-math></inline-formula> are independent Gaussian, STAF provably recovers exactly any <inline-formula> <tex-math notation="LaTeX">$\boldsymbol {x}\in \mathbb{R}^n$</tex-math></inline-formula> exponentially fast based on order of <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula> quadratic equations. STAF is also robust in the presence of additive noise of bounded support. Simulated tests involving real Gaussian <inline-formula> <tex-math notation="LaTeX">$\lbrace \boldsymbol {a}_i\rbrace$</tex-math></inline-formula> vectors demonstrate that STAF empirically reconstructs any <inline-formula><tex-math notation="LaTeX">$\boldsymbol {x}\in \mathbb{R}^n$</tex-math> </inline-formula> exactly from about <inline-formula><tex-math notation="LaTeX">$2.3n$</tex-math></inline-formula> magnitude-only measurements, outperforming state-of-the-art approaches and narrowing the gap from the information-theoretic number of equations <inline-formula><tex-math notation="LaTeX">$m=2n-1$</tex-math> </inline-formula>. Extensive experiments using synthetic data and real images corroborate markedly improved performance of STAF over existing alternatives.

[1]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[2]  Prateek Jain,et al.  Phase Retrieval Using Alternating Minimization , 2013, IEEE Transactions on Signal Processing.

[3]  Ayfer Özgür,et al.  Phase Retrieval via Incremental Truncated Wirtinger Flow , 2016, ArXiv.

[4]  Surya Ganguli,et al.  Identifying and attacking the saddle point problem in high-dimensional non-convex optimization , 2014, NIPS.

[5]  Xiaodong Li,et al.  Optimal Rates of Convergence for Noisy Sparse Phase Retrieval via Thresholded Wirtinger Flow , 2015, ArXiv.

[6]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[7]  Anastasios Kyrillidis,et al.  Finding Low-rank Solutions to Matrix Problems, Efficiently and Provably , 2016, SIAM J. Imaging Sci..

[8]  Andrea J. Goldsmith,et al.  Exact and Stable Covariance Estimation From Quadratic Sampling via Convex Programming , 2013, IEEE Transactions on Information Theory.

[9]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[10]  Yingbin Liang,et al.  Reshaped Wirtinger Flow for Solving Quadratic System of Equations , 2016, NIPS.

[11]  Yonina C. Eldar,et al.  GESPAR: Efficient Phase Retrieval of Sparse Signals , 2013, IEEE Transactions on Signal Processing.

[12]  Gang Wang,et al.  Power scheduling for Kalman filtering over lossy wireless sensor networks , 2017 .

[13]  EDWARD M. HOFSTETTER,et al.  Construction of time-limited functions with specified autocorrelation functions , 1964, IEEE Trans. Inf. Theory.

[14]  J. Miao,et al.  Extending the methodology of X-ray crystallography to allow imaging of micrometre-sized non-crystalline specimens , 1999, Nature.

[15]  R. Vershynin,et al.  A Randomized Kaczmarz Algorithm with Exponential Convergence , 2007, math/0702226.

[16]  Zhi-Quan Luo,et al.  Guaranteed Matrix Completion via Non-Convex Factorization , 2014, IEEE Transactions on Information Theory.

[17]  Yingbin Liang,et al.  Provable Non-convex Phase Retrieval with Outliers: Median TruncatedWirtinger Flow , 2016, ICML.

[18]  Gang Wang,et al.  Adaptive censoring for large-scale regressions , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[19]  Babak Hassibi,et al.  Recovery of sparse 1-D signals from the magnitudes of their Fourier transform , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[20]  Tom Goldstein,et al.  PhaseMax: Convex Phase Retrieval via Basis Pursuit , 2016, IEEE Transactions on Information Theory.

[21]  Yonina C. Eldar,et al.  Non-Convex Phase Retrieval From STFT Measurements , 2016, IEEE Transactions on Information Theory.

[22]  Nikos D. Sidiropoulos,et al.  Inexact Alternating Optimization for Phase Retrieval in the Presence of Outliers , 2016, IEEE Transactions on Signal Processing.

[23]  Nikos D. Sidiropoulos,et al.  Phase Retrieval Using Feasible Point Pursuit: Algorithms and Cramér–Rao Bound , 2015, IEEE Transactions on Signal Processing.

[24]  Yonina C. Eldar,et al.  Low-Rank Phase Retrieval , 2016, IEEE Transactions on Signal Processing.

[25]  Dan Edidin,et al.  An algebraic characterization of injectivity in phase retrieval , 2013, ArXiv.

[26]  Alexandre d'Aspremont,et al.  Phase recovery, MaxCut and complex semidefinite programming , 2012, Math. Program..

[27]  Yonina C. Eldar,et al.  Phase Retrieval from 1D Fourier Measurements: Convexity, Uniqueness, and Algorithms , 2016, IEEE Transactions on Signal Processing.

[28]  Katta G. Murty,et al.  Some NP-complete problems in quadratic and nonlinear programming , 1987, Math. Program..

[29]  Panos M. Pardalos,et al.  Quadratic programming with one negative eigenvalue is NP-hard , 1991, J. Glob. Optim..

[30]  Gang Wang,et al.  SPARTA: Sparse phase retrieval via Truncated Amplitude flow , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[31]  Emmanuel J. Candès,et al.  PhaseLift: Exact and Stable Signal Recovery from Magnitude Measurements via Convex Programming , 2011, ArXiv.

[32]  Sujay Sanghavi,et al.  The Local Convexity of Solving Systems of Quadratic Equations , 2015, 1506.07868.

[33]  Xiaodong Li,et al.  Phase Retrieval from Coded Diffraction Patterns , 2013, 1310.3240.

[34]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[35]  Anastasios Kyrillidis,et al.  Dropping Convexity for Faster Semi-definite Optimization , 2015, COLT.

[36]  Max Simchowitz,et al.  Low-rank Solutions of Linear Matrix Equations via Procrustes Flow , 2015, ICML.

[37]  Yonina C. Eldar,et al.  Phase Retrieval: An Overview of Recent Developments , 2015, ArXiv.

[38]  Yonina C. Eldar,et al.  Phase Retrieval via Matrix Completion , 2011, SIAM Rev..

[39]  R. Balan,et al.  On signal reconstruction without phase , 2006 .

[40]  Prabhu Babu,et al.  PRIME: Phase Retrieval via Majorization-Minimization , 2015, IEEE Transactions on Signal Processing.

[41]  John Wright,et al.  Complete Dictionary Recovery Over the Sphere I: Overview and the Geometric Picture , 2015, IEEE Transactions on Information Theory.

[42]  Furong Huang,et al.  Escaping From Saddle Points - Online Stochastic Gradient for Tensor Decomposition , 2015, COLT.

[43]  Deanna Needell,et al.  Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm , 2013, Mathematical Programming.

[44]  Gang Wang,et al.  Solving Random Systems of Quadratic Equations via Truncated Generalized Gradient Flow , 2016, NIPS.

[45]  Constantine Caramanis,et al.  A Convex Formulation for Mixed Regression: Near Optimal Rates in the Face of Noise , 2013, ArXiv.

[46]  Bing Gao,et al.  Phaseless Recovery Using the Gauss–Newton Method , 2016, IEEE Transactions on Signal Processing.

[47]  Rick P. Millane,et al.  Phase retrieval in crystallography and optics , 1990 .

[48]  Gang Wang,et al.  Online reconstruction from big data via compressive censoring , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[49]  H. Sahinoglou,et al.  On phase retrieval of finite-length sequences using the initial time sample , 1991 .

[50]  Yuxin Chen,et al.  Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems , 2015, NIPS.

[51]  Tong Zhang,et al.  Accelerating Stochastic Gradient Descent using Predictive Variance Reduction , 2013, NIPS.

[52]  J R Fienup,et al.  Phase retrieval algorithms: a comparison. , 1982, Applied optics.

[53]  Ohad Shamir,et al.  Fast Stochastic Algorithms for SVD and PCA: Convergence Properties and Convexity , 2015, ICML.

[54]  John Wright,et al.  A Geometric Analysis of Phase Retrieval , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[55]  Irène Waldspurger,et al.  Phase Retrieval With Random Gaussian Sensing Vectors by Alternating Projections , 2016, IEEE Transactions on Information Theory.

[56]  Daniel Pérez Palomar,et al.  Undersampled Sparse Phase Retrieval via Majorization–Minimization , 2016, IEEE Transactions on Signal Processing.

[57]  Yonina C. Eldar,et al.  Solving Systems of Random Quadratic Equations via Truncated Amplitude Flow , 2016, IEEE Transactions on Information Theory.

[58]  Yuejie Chi,et al.  Reshaped Wirtinger Flow and Incremental Algorithm for Solving Quadratic System of Equations , 2016 .

[59]  O. Bunk,et al.  Diffractive imaging for periodic samples: retrieving one-dimensional concentration profiles across microfluidic channels. , 2007, Acta crystallographica. Section A, Foundations of crystallography.

[60]  R. Gerchberg.  A practical algorithm for the determination of phase from image and diffraction plane pictures , 1972 .

[61]  Ke Wei.  Solving systems of phaseless equations via Kaczmarz methods: a proof of concept study , 2015 .

[62]  Gang Wang,et al.  Sparse Phase Retrieval via Truncated Amplitude Flow , 2016, IEEE Transactions on Signal Processing.

[63]  Martin Vetterli,et al.  Phase Retrieval for Sparse Signals: Uniqueness Conditions , 2013, ArXiv.

[64]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.