Global Geometry of Multichannel Sparse Blind Deconvolution on the Sphere

Multichannel blind deconvolution is the problem of recovering an unknown signal $f$ and multiple unknown channels $x_i$ from convolutional measurements $y_i=x_i \circledast f$ ($i=1,2,\dots,N$). We consider the case where the $x_i$'s are sparse, and convolution with $f$ is invertible. Our nonconvex optimization formulation solves for a filter $h$ on the unit sphere that produces sparse output $y_i\circledast h$. Under some technical assumptions, we show that all local minima of the objective function correspond to the inverse filter of $f$ up to an inherent sign and shift ambiguity, and all saddle points have strictly negative curvatures. This geometric structure allows successful recovery of $f$ and $x_i$ using a simple manifold gradient descent algorithm with random initialization. Our theoretical findings are complemented by numerical experiments, which demonstrate superior performance of the proposed approach over the previous methods.

[1]  Thomas Kailath,et al.  Direction of arrival estimation by eigenstructure methods with unknown sensor gain and phase , 1985, ICASSP '85. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[2]  John Wright,et al.  Complete dictionary recovery over the sphere , 2015, 2015 International Conference on Sampling Theory and Applications (SampTA).

[3]  Sumit Roy,et al.  Self-calibration of linear equi-spaced (LES) arrays , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[4]  Chrysostomos L. Nikias,et al.  EVAM: an eigenvector-based algorithm for multichannel blind deconvolution of input colored signals , 1995, IEEE Trans. Signal Process..

[5]  Karl J. Friston,et al.  Modeling regional and psychophysiologic interactions in fMRI: the importance of hemodynamic deconvolution , 2003, NeuroImage.

[6]  Shengli Zhou,et al.  Sparse channel estimation for multicarrier underwater acoustic communication: From subspace methods to compressed sensing , 2009, OCEANS 2009-EUROPE.

[7]  A. Montanari,et al.  The landscape of empirical risk for nonconvex losses , 2016, The Annals of Statistics.

[8]  Subhasis Chaudhuri,et al.  Blind Image Deconvolution , 2014, Springer International Publishing.

[9]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[10]  John Wright,et al.  Complete Dictionary Recovery Over the Sphere II: Recovery by Riemannian Trust-Region Method , 2015, IEEE Transactions on Information Theory.

[11]  Zeyuan Allen-Zhu Natasha: Faster Stochastic Non-Convex Optimization via Strongly Non-Convex Parameter , 2017 .

[12]  Lang Tong,et al.  A new approach to blind identification and equalization of multipath channels , 1991, [1991] Conference Record of the Twenty-Fifth Asilomar Conference on Signals, Systems & Computers.

[13]  Thomas Strohmer,et al.  Self-calibration and biconvex compressive sensing , 2015, ArXiv.

[14]  Justin Romberg,et al.  Fast and Guaranteed Blind Multichannel Deconvolution Under a Bilinear System Model , 2016, IEEE Transactions on Information Theory.

[15]  Rémi Gribonval,et al.  Convex Optimization Approaches for Blind Sensor Calibration Using Sparsity , 2013, IEEE Transactions on Signal Processing.

[16]  A. Nehorai,et al.  Deconvolution methods for 3-D fluorescence microscopy images , 2006, IEEE Signal Processing Magazine.

[17]  Yanjun Li,et al.  Blind Gain and Phase Calibration via Sparse Spectral Methods , 2017, IEEE Transactions on Information Theory.

[18]  Seungyong Lee,et al.  Fast motion deblurring , 2009, ACM Trans. Graph..

[19]  Liming Wang,et al.  Blind Deconvolution From Multiple Sparse Inputs , 2016, IEEE Signal Processing Letters.

[20]  John Wright,et al.  On the Global Geometry of Sphere-Constrained Sparse Blind Deconvolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Georgios Piliouras,et al.  Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions , 2016, ITCS.

[22]  Xiaodong Li,et al.  Rapid, Robust, and Reliable Blind Deconvolution via Nonconvex Optimization , 2016, Applied and Computational Harmonic Analysis.

[23]  Karim G. Sabra,et al.  Blind deconvolution in ocean waveguides using artificial time reversal , 2004 .

[24]  Justin K. Romberg,et al.  Blind Deconvolution Using Convex Programming , 2012, IEEE Transactions on Information Theory.

[25]  Yoram Bresler,et al.  FIR perfect signal reconstruction from multiple convolutions: minimum deconvolver orders , 1998, IEEE Trans. Signal Process..

[26]  Michael I. Jordan,et al.  How to Escape Saddle Points Efficiently , 2017, ICML.

[27]  Yanjun Li,et al.  Optimal Sample Complexity for Blind Gain and Phase Calibration , 2015, IEEE Transactions on Signal Processing.

[28]  Levent Tunçel,et al.  Optimization algorithms on matrix manifolds , 2009, Math. Comput..

[29]  Felix Krahmer,et al.  Spectral Methods for Passive Imaging: Non-asymptotic Performance and Robustness , 2017, SIAM J. Imaging Sci..

[30]  Michael J Rust,et al.  Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM) , 2006, Nature Methods.

[31]  Yonina C. Eldar,et al.  Sensor Calibration for Off-the-Grid Spectral Estimation , 2017, Applied and Computational Harmonic Analysis.

[32]  Wen Huang,et al.  Blind Deconvolution by a Steepest Descent Algorithm on a Quotient Manifold , 2017, SIAM J. Imaging Sci..

[33]  Joel A. Tropp,et al.  User-Friendly Tail Bounds for Sums of Random Matrices , 2010, Found. Comput. Math..

[34]  John Wright,et al.  A Geometric Analysis of Phase Retrieval , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[35]  Dong Liang,et al.  Image reconstruction from phased-array data based on multichannel blind deconvolution. , 2015, Magnetic resonance imaging.

[36]  Yanning Zhang,et al.  Multi-image Blind Deblurring Using a Coupled Adaptive Sparse Prior , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Thomas Strohmer Four short stories about Toeplitz matrix calculations , 2000 .

[38]  T. Kailath,et al.  A least-squares approach to blind channel identification , 1995, IEEE Trans. Signal Process..

[39]  Yanjun Li,et al.  Identifiability in Bilinear Inverse Problems With Applications to Subspace or Sparsity-Constrained Blind Gain and Phase Calibration , 2017, IEEE Transactions on Information Theory.

[40]  Kjetil F. Kaaresen,et al.  Multichannel blind deconvolution of seismic signals , 1998 .

[41]  Yanjun Li,et al.  Blind Recovery of Sparse Signals From Subsampled Convolution , 2015, IEEE Transactions on Information Theory.

[42]  L. Balzano,et al.  Blind Calibration of Sensor Networks , 2007, 2007 6th International Symposium on Information Processing in Sensor Networks.

[43]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[44]  Xiao-Tong Yuan,et al.  Truncated power method for sparse eigenvalue problems , 2011, J. Mach. Learn. Res..

[45]  Michael I. Jordan,et al.  Gradient Descent Only Converges to Minimizers , 2016, COLT.

[46]  Yuejie Chi,et al.  Guaranteed Blind Sparse Spikes Deconvolution via Lifting and Convex Optimization , 2015, IEEE Journal of Selected Topics in Signal Processing.

[47]  Thomas Strohmer,et al.  Self-Calibration via Linear Least Squares , 2016, ArXiv.

[48]  Zeyuan Allen-Zhu,et al.  Natasha 2: Faster Non-Convex Optimization Than SGD , 2017, NeurIPS.

[49]  Justin Romberg,et al.  Multichannel myopic deconvolution in underwater acoustic channels via low-rank recovery. , 2017, The Journal of the Acoustical Society of America.

[50]  John Wright,et al.  Using negative curvature in solving nonlinear programs , 2017, Comput. Optim. Appl..

[51]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[52]  Li Xu,et al.  Unnatural L0 Sparse Representation for Natural Image Deblurring , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[53]  Eric Moulines,et al.  Subspace methods for the blind identification of multichannel FIR filters , 1995, IEEE Trans. Signal Process..

[54]  Michael I. Jordan,et al.  First-order methods almost always avoid saddle points: The case of vanishing step-sizes , 2019, NeurIPS.

[55]  P. Absil,et al.  Erratum to: ``Global rates of convergence for nonconvex optimization on manifolds'' , 2016, IMA Journal of Numerical Analysis.

[56]  Yanjun Li,et al.  Identifiability in Blind Deconvolution With Subspace or Sparsity Constraints , 2015, IEEE Transactions on Information Theory.

[57]  John Wright,et al.  Structured Local Optima in Sparse Blind Deconvolution , 2018, IEEE Transactions on Information Theory.

[58]  J. Lippincott-Schwartz,et al.  Imaging Intracellular Fluorescent Proteins at Nanometer Resolution , 2006, Science.

[59]  Frédo Durand,et al.  Understanding Blind Deconvolution Algorithms , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  John Wright,et al.  Complete Dictionary Recovery Over the Sphere I: Overview and the Geometric Picture , 2015, IEEE Transactions on Information Theory.

[61]  Furong Huang,et al.  Escaping From Saddle Points - Online Stochastic Gradient for Tensor Decomposition , 2015, COLT.

[62]  Yanjun Li,et al.  Identifiability and Stability in Blind Deconvolution Under Minimal Assumptions , 2015, IEEE Transactions on Information Theory.

[63]  X. Zhuang,et al.  Statistical deconvolution for superresolution fluorescence microscopy. , 2012, Biophysical journal.

[64]  L. Tong,et al.  Multichannel blind identification: from subspace to maximum likelihood methods , 1998, Proc. IEEE.

[65]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[66]  Tengyu Ma,et al.  Finding approximate local minima faster than gradient descent , 2016, STOC.