A Nonconvex Approach for Exact and Efficient Multichannel Sparse Blind Deconvolution

We study the multi-channel sparse blind deconvolution (MCS-BD) problem, whose task is to simultaneously recover a kernel $\mathbf a$ and multiple sparse inputs $\{\mathbf x_i\}_{i=1}^p$ from their circulant convolution $\mathbf y_i = \mb a \circledast \mb x_i $ ($i=1,\cdots,p$). We formulate the task as a nonconvex optimization problem over the sphere. Under mild statistical assumptions of the data, we prove that the vanilla Riemannian gradient descent (RGD) method, with random initializations, provably recovers both the kernel $\mathbf a$ and the signals $\{\mathbf x_i\}_{i=1}^p$ up to a signed shift ambiguity. In comparison with state-of-the-art results, our work shows significant improvements in terms of sample complexity and computational efficiency. Our theoretical results are corroborated by numerical experiments, which demonstrate superior performance of the proposed approach over the previous methods on both synthetic and real datasets.

[1]  Dong Liang,et al.  Image reconstruction from phased-array data based on multichannel blind deconvolution. , 2015, Magnetic resonance imaging.

[2]  Jeffrey A. Fessler,et al.  Convolutional Dictionary Learning: Acceleration and Convergence , 2017, IEEE Transactions on Image Processing.

[3]  Yuejie Chi,et al.  Manifold Gradient Descent Solves Multi-Channel Sparse Blind Deconvolution Provably and Efficiently , 2019, IEEE Transactions on Information Theory.

[4]  Thomas Strohmer,et al.  Self-Calibration and Bilinear Inverse Problems via Linear Least Squares , 2016, SIAM J. Imaging Sci..

[5]  David Pfau,et al.  Simultaneous Denoising, Deconvolution, and Demixing of Calcium Imaging Data , 2016, Neuron.

[6]  Justin Romberg,et al.  Fast and Guaranteed Blind Multichannel Deconvolution Under a Bilinear System Model , 2016, IEEE Transactions on Information Theory.

[7]  Lei Zhu,et al.  Faster STORM using compressed sensing , 2012, Nature Methods.

[8]  Frédo Durand,et al.  Understanding Blind Deconvolution Algorithms , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  JainPrateek,et al.  Non-convex Optimization for Machine Learning , 2017 .

[10]  John Wright,et al.  Complete Dictionary Recovery Over the Sphere I: Overview and the Geometric Picture , 2015, IEEE Transactions on Information Theory.

[11]  Yu Bai,et al.  Subgradient Descent Learns Orthogonal Dictionaries , 2018, ICLR.

[12]  Daniel P. Robinson,et al.  Dual Principal Component Pursuit: Improved Analysis and Efficient Algorithms , 2018, NeurIPS.

[13]  Felix Krahmer,et al.  Optimal Injectivity Conditions for Bilinear Inverse Problems with Applications to Identifiability of Deconvolution Problems , 2016, SIAM J. Appl. Algebra Geom..

[14]  John Wright,et al.  On the Global Geometry of Sphere-Constrained Sparse Blind Deconvolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Xiaodong Li,et al.  Rapid, Robust, and Reliable Blind Deconvolution via Nonconvex Optimization , 2016, Applied and Computational Harmonic Analysis.

[16]  Yanjun Li,et al.  Blind Gain and Phase Calibration via Sparse Spectral Methods , 2017, IEEE Transactions on Information Theory.

[17]  Liming Wang,et al.  Blind Deconvolution From Multiple Sparse Inputs , 2016, IEEE Signal Processing Letters.

[18]  T. Coleman,et al.  The null space problem I. complexity , 1986 .

[19]  Daniel P. Robinson,et al.  Noisy Dual Principal Component Pursuit , 2019, ICML.

[20]  Jean-Louis Goffin,et al.  On convergence rates of subgradient optimization methods , 1977, Math. Program..

[21]  Prateek Jain,et al.  Non-convex Optimization for Machine Learning , 2017, Found. Trends Mach. Learn..

[22]  Yanjun Li,et al.  Identifiability in Blind Deconvolution With Subspace or Sparsity Constraints , 2015, IEEE Transactions on Information Theory.

[23]  John Wright,et al.  Structured Local Optima in Sparse Blind Deconvolution , 2018, IEEE Transactions on Information Theory.

[24]  J. Lippincott-Schwartz,et al.  Imaging Intracellular Fluorescent Proteins at Nanometer Resolution , 2006, Science.

[25]  Eric Moulines,et al.  Subspace methods for the blind identification of multichannel FIR filters , 1995, IEEE Trans. Signal Process..

[26]  Nathan Srebro,et al.  Implicit Regularization in Matrix Factorization , 2017, 2018 Information Theory and Applications Workshop (ITA).

[27]  Zhihui Zhu,et al.  Finding the Sparsest Vectors in a Subspace: Theory, Algorithms, and Applications , 2020, ArXiv.

[28]  Yuxin Chen,et al.  Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview , 2018, IEEE Transactions on Signal Processing.

[29]  John Wright,et al.  Complete Dictionary Recovery Over the Sphere II: Recovery by Riemannian Trust-Region Method , 2015, IEEE Transactions on Information Theory.

[30]  Yuejie Chi,et al.  Guaranteed Blind Sparse Spikes Deconvolution via Lifting and Convex Optimization , 2015, IEEE Journal of Selected Topics in Signal Processing.

[31]  Thomas Strohmer,et al.  Self-calibration and biconvex compressive sensing , 2015, ArXiv.

[32]  John Wright,et al.  Efficient Dictionary Learning with Gradient Descent , 2018, ICML.

[33]  Dong Liang,et al.  Image reconstruction from phased-array MRI data based on multichannel blind deconvolution , 2010, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[34]  Sjoerd Dirksen,et al.  Tail bounds via generic chaining , 2013, ArXiv.

[35]  Dmitriy Drusvyatskiy,et al.  Low-Rank Matrix Recovery with Composite Optimization: Good Conditioning and Rapid Convergence , 2019, Found. Comput. Math..

[36]  Yudong Chen,et al.  Harnessing Structures in Big Data via Guaranteed Low-Rank Matrix Estimation: Recent Theory and Fast Algorithms via Convex and Nonconvex Optimization , 2018, IEEE Signal Processing Magazine.

[37]  Gilad Lerman,et al.  An Overview of Robust Subspace Recovery , 2018, Proceedings of the IEEE.

[38]  Laurent Demanet,et al.  Leveraging Diversity and Sparsity in Blind Deconvolution , 2016, IEEE Transactions on Information Theory.

[39]  J. Pesquet,et al.  N ov 2 01 4 Euclid in a Taxicab : Sparse Blind Deconvolution with Smoothed l 1 / l 2 Regularization , 2014 .

[40]  Anders P. Eriksson,et al.  Fast Convolutional Sparse Coding , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Holger Rauhut,et al.  A Mathematical Introduction to Compressive Sensing , 2013, Applied and Numerical Harmonic Analysis.

[42]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[43]  Felix Krahmer,et al.  Spectral Methods for Passive Imaging: Non-asymptotic Performance and Robustness , 2017, SIAM J. Imaging Sci..

[44]  Michael J Rust,et al.  Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM) , 2006, Nature Methods.

[45]  John Wright,et al.  Geometry and Symmetry in Short-and-Sparse Deconvolution , 2019, ICML.

[46]  Brendt Wohlberg,et al.  Convolutional Dictionary Learning: A Comparative Review and New Algorithms , 2017, IEEE Transactions on Computational Imaging.

[47]  João Marcos Travassos Romano,et al.  A fast algorithm for sparse multichannel blind deconvolution , 2016 .

[48]  Liam Paninski,et al.  Fast online deconvolution of calcium imaging data , 2016, PLoS Comput. Biol..

[49]  A. Nehorai,et al.  Deconvolution methods for 3-D fluorescence microscopy images , 2006, IEEE Signal Processing Magazine.

[50]  Stephen P. Boyd,et al.  Disciplined Convex Programming , 2006 .

[51]  Yanning Zhang,et al.  Multi-image Blind Deblurring Using a Coupled Adaptive Sparse Prior , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Yi Ma,et al.  Complete Dictionary Learning via 𝓁4-Norm Maximization over the Orthogonal Group , 2019, J. Mach. Learn. Res..

[53]  Augustin Cosse,et al.  A note on the blind deconvolution of multiple sparse signals from unknown subspaces , 2017, Optical Engineering + Applications.

[54]  Guorong Wu,et al.  A blind deconvolution approach to recover effective connectivity brain networks from resting state fMRI data , 2012, Medical Image Anal..

[55]  John Wright,et al.  Finding a Sparse Vector in a Subspace: Linear Sparsity Using Alternating Directions , 2014, IEEE Transactions on Information Theory.

[56]  A. Small,et al.  Fluorophore localization algorithms for super-resolution microscopy , 2014, Nature Methods.

[57]  Dmitriy Drusvyatskiy,et al.  Subgradient Methods for Sharp Weakly Convex Functions , 2018, Journal of Optimization Theory and Applications.

[58]  E. Giné,et al.  Decoupling: From Dependence to Independence , 1998 .

[59]  Anthony Man-Cho So,et al.  Incremental Methods for Weakly Convex Optimization , 2019, ArXiv.

[60]  Eero P. Simoncelli,et al.  A blind sparse deconvolution method for neural spike identification , 2011, NIPS.

[61]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[62]  Yuxin Chen,et al.  Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution , 2017, Found. Comput. Math..

[63]  Justin K. Romberg,et al.  Blind Deconvolution Using Convex Programming , 2012, IEEE Transactions on Information Theory.

[64]  René Vidal,et al.  Dual Principal Component Pursuit , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[65]  Robert E. Mahony,et al.  Optimization Algorithms on Matrix Manifolds , 2007 .

[66]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[67]  Peyman Milanfar,et al.  Robust Multichannel Blind Deconvolution via Fast Alternating Minimization , 2012, IEEE Transactions on Image Processing.

[68]  Michael D. Mason,et al.  Ultra-high resolution imaging by fluorescence photoactivation localization microscopy. , 2006, Biophysical journal.

[69]  Holger Rauhut,et al.  Suprema of Chaos Processes and the Restricted Isometry Property , 2012, ArXiv.

[70]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[71]  Yanjun Li,et al.  Global Geometry of Multichannel Sparse Blind Deconvolution on the Sphere , 2018, NeurIPS.

[72]  Yonina C. Eldar,et al.  Convolutional Phase Retrieval , 2017, NIPS.

[73]  S.C. Douglas,et al.  Multichannel blind deconvolution and equalization using the natural gradient , 1997, First IEEE Signal Processing Workshop on Signal Processing Advances in Wireless Communications.

[74]  Yoram Bresler,et al.  FIR perfect signal reconstruction from multiple convolutions: minimum deconvolver orders , 1998, IEEE Trans. Signal Process..

[75]  Xiao Li,et al.  Nonconvex Robust Low-rank Matrix Recovery , 2018, SIAM J. Optim..

[76]  Jingdong Chen,et al.  Blind channel identification for speech dereverberation using l1-norm sparse learning , 2007, NIPS.

[77]  Robert M. Gray,et al.  Toeplitz and Circulant Matrices: A Review , 2005, Found. Trends Commun. Inf. Theory.

[78]  S. Holden,et al.  DAOSTORM: an algorithm for high- density super-resolution microscopy , 2011, Nature Methods.

[79]  Yuxin Chen,et al.  Gradient descent with random initialization: fast global convergence for nonconvex phase retrieval , 2018, Mathematical Programming.

[80]  M. Talagrand Upper and Lower Bounds for Stochastic Processes: Modern Methods and Classical Problems , 2014 .

[81]  Karl J. Friston,et al.  Modeling regional and psychophysiologic interactions in fMRI: the importance of hemodynamic deconvolution , 2003, NeuroImage.

[82]  M. Ferris,et al.  Weak sharp minima in mathematical programming , 1993 .

[83]  Pengcheng Zhou,et al.  Short-and-Sparse Deconvolution - A Geometric Approach , 2019, ICLR.

[84]  Yanjun Li,et al.  Multichannel Sparse Blind Deconvolution on the Sphere , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[85]  Yanjun Li,et al.  Bilinear inverse problems with sparsity: optimal identifiability conditions and efficient recovery , 2018 .

[86]  Justin Romberg,et al.  Multichannel myopic deconvolution in underwater acoustic channels via low-rank recovery. , 2017, The Journal of the Acoustical Society of America.

[87]  Jean-Christophe Pesquet,et al.  Euclid in a Taxicab: Sparse Blind Deconvolution with Smoothed ${\ell _1}/{\ell _2}$ Regularization , 2014, IEEE Signal Processing Letters.

[88]  T. Kailath,et al.  A least-squares approach to blind channel identification , 1995, IEEE Trans. Signal Process..

[89]  Yanjun Li,et al.  Identifiability in Bilinear Inverse Problems With Applications to Subspace or Sparsity-Constrained Blind Gain and Phase Calibration , 2017, IEEE Transactions on Information Theory.

[90]  Ruslan Salakhutdinov,et al.  Geometry of Optimization and Implicit Regularization in Deep Learning , 2017, ArXiv.

[91]  Kjetil F. Kaaresen,et al.  Multichannel blind deconvolution of seismic signals , 1998 .