Multichannel Sparse Blind Deconvolution on the Sphere

Multichannel blind deconvolution is the problem of recovering an unknown signal f and multiple unknown channels xi from convolutional measurements yi = xi ⊛ f (i = 1, 2, …, N). We consider the case where the xi’s are sparse, and convolution with f is invertible. Our nonconvex optimization formulation solves for a filter h on the unit sphere that produces sparse output yi ⊛ h. Under some technical assumptions, we show that all local minima of the objective function correspond to the inverse filter of f up to an inherent sign and shift ambiguity, and all saddle points have strictly negative curvatures. This geometric structure allows successful recovery of f and xi using a simple manifold gradient descent algorithm with random initialization. Our theoretical findings are complemented by numerical experiments, which demonstrate superior performance of the proposed approach over the previous methods.

[1]  T. Kailath,et al.  A least-squares approach to blind channel identification , 1995, IEEE Trans. Signal Process..

[2]  Yanjun Li,et al.  Identifiability in Bilinear Inverse Problems With Applications to Subspace or Sparsity-Constrained Blind Gain and Phase Calibration , 2017, IEEE Transactions on Information Theory.

[3]  X. Zhuang,et al.  Statistical deconvolution for superresolution fluorescence microscopy. , 2012, Biophysical journal.

[4]  Kjetil F. Kaaresen,et al.  Multichannel blind deconvolution of seismic signals , 1998 .

[5]  L. Tong,et al.  Multichannel blind identification: from subspace to maximum likelihood methods , 1998, Proc. IEEE.

[6]  Thomas Kailath,et al.  Direction of arrival estimation by eigenstructure methods with unknown sensor gain and phase , 1985, ICASSP '85. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[7]  Yanning Zhang,et al.  Multi-image Blind Deblurring Using a Coupled Adaptive Sparse Prior , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Thomas Strohmer Four short stories about Toeplitz matrix calculations , 2000 .

[9]  John Wright,et al.  On the Global Geometry of Sphere-Constrained Sparse Blind Deconvolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Georgios Piliouras,et al.  Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions , 2016, ITCS.

[11]  Xiaodong Li,et al.  Rapid, Robust, and Reliable Blind Deconvolution via Nonconvex Optimization , 2016, Applied and Computational Harmonic Analysis.

[12]  P. Absil,et al.  Erratum to: ``Global rates of convergence for nonconvex optimization on manifolds'' , 2016, IMA Journal of Numerical Analysis.

[13]  John Wright,et al.  Structured Local Optima in Sparse Blind Deconvolution , 2018, IEEE Transactions on Information Theory.

[14]  J. Lippincott-Schwartz,et al.  Imaging Intracellular Fluorescent Proteins at Nanometer Resolution , 2006, Science.

[15]  Zeyuan Allen-Zhu,et al.  Natasha: Faster Non-Convex Stochastic Optimization via Strongly Non-Convex Parameter , 2017, ICML.

[16]  John Wright,et al.  Using negative curvature in solving nonlinear programs , 2017, Comput. Optim. Appl..

[17]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[18]  Zeyuan Allen-Zhu,et al.  Natasha 2: Faster Non-Convex Optimization Than SGD , 2017, NeurIPS.

[19]  Michael I. Jordan,et al.  Stochastic Gradient Descent Escapes Saddle Points Efficiently , 2019, ArXiv.

[20]  Michael I. Jordan,et al.  On Nonconvex Optimization for Machine Learning , 2019, J. ACM.

[21]  Justin Romberg,et al.  Multichannel myopic deconvolution in underwater acoustic channels via low-rank recovery. , 2017, The Journal of the Acoustical Society of America.

[22]  Justin Romberg,et al.  Fast and Guaranteed Blind Multichannel Deconvolution Under a Bilinear System Model , 2016, IEEE Transactions on Information Theory.

[23]  Yuxin Chen,et al.  Gradient descent with random initialization: fast global convergence for nonconvex phase retrieval , 2018, Mathematical Programming.

[24]  Sumit Roy,et al.  Self-calibration of linear equi-spaced (LES) arrays , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[25]  Karl J. Friston,et al.  Modeling regional and psychophysiologic interactions in fMRI: the importance of hemodynamic deconvolution , 2003, NeuroImage.

[26]  Karim G. Sabra,et al.  Blind deconvolution in ocean waveguides using artificial time reversal , 2004 .

[27]  Wen Huang,et al.  Blind Deconvolution by a Steepest Descent Algorithm on a Quotient Manifold , 2017, SIAM J. Imaging Sci..

[28]  Yu Bai,et al.  Subgradient Descent Learns Orthogonal Dictionaries , 2018, ICLR.

[29]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Yoram Bresler,et al.  FIR perfect signal reconstruction from multiple convolutions: minimum deconvolver orders , 1998, IEEE Trans. Signal Process..

[31]  Michael I. Jordan,et al.  How to Escape Saddle Points Efficiently , 2017, ICML.

[32]  Tengyu Ma,et al.  Finding approximate local minima faster than gradient descent , 2016, STOC.

[33]  Levent Tunçel,et al.  Optimization algorithms on matrix manifolds , 2009, Math. Comput..

[34]  Felix Krahmer,et al.  Spectral Methods for Passive Imaging: Non-asymptotic Performance and Robustness , 2017, SIAM J. Imaging Sci..

[35]  Michael J Rust,et al.  Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM) , 2006, Nature Methods.

[36]  Yonina C. Eldar,et al.  Sensor Calibration for Off-the-Grid Spectral Estimation , 2017, Applied and Computational Harmonic Analysis.

[37]  Rémi Gribonval,et al.  Convex Optimization Approaches for Blind Sensor Calibration Using Sparsity , 2013, IEEE Transactions on Signal Processing.

[38]  A. Nehorai,et al.  Deconvolution methods for 3-D fluorescence microscopy images , 2006, IEEE Signal Processing Magazine.

[39]  John Wright,et al.  A Geometric Analysis of Phase Retrieval , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[40]  Dong Liang,et al.  Image reconstruction from phased-array data based on multichannel blind deconvolution. , 2015, Magnetic resonance imaging.

[41]  Nicholas J. Higham,et al.  Functions of matrices - theory and computation , 2008 .

[42]  Justin K. Romberg,et al.  Blind Deconvolution Using Convex Programming , 2012, IEEE Transactions on Information Theory.

[43]  Joel A. Tropp,et al.  User-Friendly Tail Bounds for Sums of Random Matrices , 2010, Found. Comput. Math..

[44]  John Wright,et al.  Complete dictionary recovery over the sphere , 2015, 2015 International Conference on Sampling Theory and Applications (SampTA).

[45]  Michael I. Jordan,et al.  First-order methods almost always avoid saddle points: The case of vanishing step-sizes , 2019, NeurIPS.

[46]  Chrysostomos L. Nikias,et al.  EVAM: an eigenvector-based algorithm for multichannel blind deconvolution of input colored signals , 1995, IEEE Trans. Signal Process..

[47]  A. Montanari,et al.  The landscape of empirical risk for nonconvex losses , 2016, The Annals of Statistics.

[48]  Deepa Kundur,et al.  Blind image deconvolution , 1996, IEEE Signal Process. Mag..

[49]  Yanjun Li,et al.  Blind Gain and Phase Calibration via Sparse Spectral Methods , 2017, IEEE Transactions on Information Theory.

[50]  Seungyong Lee,et al.  Fast motion deblurring , 2009, ACM Trans. Graph..

[51]  Liming Wang,et al.  Blind Deconvolution From Multiple Sparse Inputs , 2016, IEEE Signal Processing Letters.

[52]  John Wright,et al.  Complete Dictionary Recovery Over the Sphere II: Recovery by Riemannian Trust-Region Method , 2015, IEEE Transactions on Information Theory.

[53]  Lang Tong,et al.  A new approach to blind identification and equalization of multipath channels , 1991, [1991] Conference Record of the Twenty-Fifth Asilomar Conference on Signals, Systems & Computers.

[54]  Thomas Strohmer,et al.  Self-calibration and biconvex compressive sensing , 2015, ArXiv.

[55]  Thomas Strohmer,et al.  Self-Calibration and Bilinear Inverse Problems via Linear Least Squares , 2016, SIAM J. Imaging Sci..

[56]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[57]  Yanjun Li,et al.  Global Geometry of Multichannel Sparse Blind Deconvolution on the Sphere , 2018, NeurIPS.

[58]  L. Balzano,et al.  Blind Calibration of Sensor Networks , 2007, 2007 6th International Symposium on Information Processing in Sensor Networks.

[59]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[60]  Xiao-Tong Yuan,et al.  Truncated power method for sparse eigenvalue problems , 2011, J. Mach. Learn. Res..

[61]  Michael I. Jordan,et al.  Gradient Descent Only Converges to Minimizers , 2016, COLT.

[62]  Yuejie Chi,et al.  Guaranteed Blind Sparse Spikes Deconvolution via Lifting and Convex Optimization , 2015, IEEE Journal of Selected Topics in Signal Processing.

[63]  Ehud Weinstein,et al.  New criteria for blind deconvolution of nonminimum phase systems (channels) , 1990, IEEE Trans. Inf. Theory.

[64]  Frédo Durand,et al.  Understanding Blind Deconvolution Algorithms , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[65]  John Wright,et al.  Complete Dictionary Recovery Over the Sphere I: Overview and the Geometric Picture , 2015, IEEE Transactions on Information Theory.

[66]  Furong Huang,et al.  Escaping From Saddle Points - Online Stochastic Gradient for Tensor Decomposition , 2015, COLT.

[67]  Yanjun Li,et al.  Blind Recovery of Sparse Signals From Subsampled Convolution , 2015, IEEE Transactions on Information Theory.

[68]  Li Xu,et al.  Unnatural L0 Sparse Representation for Natural Image Deblurring , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[69]  Eric Moulines,et al.  Subspace methods for the blind identification of multichannel FIR filters , 1995, IEEE Trans. Signal Process..

[70]  Shengli Zhou,et al.  Sparse channel estimation for multicarrier underwater acoustic communication: From subspace methods to compressed sensing , 2009, OCEANS 2009-EUROPE.