Convolutional Dictionary Learning With Grid Refinement

Given a continuous-domain signal that can be modeled as the superposition of localized events from multiple sources, the goal of Convolutional Dictionary Learning (CDL) is to identify the location of the events–by Convolutional Sparse Coding (CSC)–and learn the template for each source–through a Convolutional Dictionary Update (CDU) step. In practice, because we observe samples of the continuous-domain signal on a discrete grid, classical CSC methods can only produce estimates of the locations of the events on this grid, which degrades the performance of the CDU step. We introduce a CDL framework that significantly reduces the errors arising from performing the estimation on the grid. Specifically, we construct an expanded dictionary that comprises, not only discrete shifts of the templates, but also variants shifted by a non-integer amount and smoothly interpolated, that enable CSC and CDU to operate at a finer resolution than that of the original sampling grid. We term this approach CDL with grid refinement. For CSC, we develop a novel computationally efficient algorithm, termed Convolutional Orthogonal Matching Pursuit with an interpolated dictionary (COMP-INTERP). We use simulated data to compare COMP-INTERP to state-of-the-art CSC algorithms for estimating off-the-grid events, and demonstrate that it 1) achieves a competitive level of accuracy, and 2) is one order of magnitude faster. For CDU, we derive a novel procedure to update the templates given sparse codes that can occur both on and off the sampling grid. We also show that 3) dictionary update with the overcomplete dictionary yields more accurate templates. Finally, we demonstrate the competitive performance of the algorithms in two applications, namely spike sorting and super-resolution microscopy.

[1]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[2]  Kenneth D. Harris,et al.  Fast and accurate spike sorting of high-channel count probes with KiloSort , 2016, NIPS.

[3]  Jonathan W. Pillow,et al.  Inferring sparse representations of continuous signals with continuous orthogonal matching pursuit , 2014, NIPS.

[4]  J. Lippincott-Schwartz,et al.  Imaging Intracellular Fluorescent Proteins at Nanometer Resolution , 2006, Science.

[5]  Wenjing Liao,et al.  Coherence Pattern-Guided Compressive Sensing with Unresolved Grids , 2011, SIAM J. Imaging Sci..

[6]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[7]  Terrence J. Sejnowski,et al.  Coding Time-Varying Signals Using Sparse, Shift-Invariant Representations , 1998, NIPS.

[8]  Holger Rauhut,et al.  Random Sampling of Sparse Trigonometric Polynomials, II. Orthogonal Matching Pursuit versus Basis Pursuit , 2008, Found. Comput. Math..

[9]  Pierre Vandergheynst,et al.  Fast orthogonal sparse approximation algorithms over local dictionaries , 2011, Signal Process..

[10]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[11]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[12]  Eero P. Simoncelli,et al.  Journal of Neuroscience Methods , 2022 .

[13]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[14]  Emmanuel Soubies,et al.  The sliding Frank–Wolfe algorithm and its application to super-resolution microscopy , 2018, Inverse Problems.

[15]  Suliana Manley,et al.  Quantitative evaluation of software packages for single-molecule localization microscopy , 2015, Nature Methods.

[16]  Brendt Wohlberg,et al.  Convolutional Dictionary Learning: A Comparative Review and New Algorithms , 2017, IEEE Transactions on Computational Imaging.

[17]  Benjamin Recht,et al.  The alternating descent conditional gradient method for sparse inverse problems , 2015, 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[18]  Prateek Jain,et al.  Learning Sparsely Used Overcomplete Dictionaries via Alternating Minimization , 2013, SIAM J. Optim..

[19]  Richard A. Andersen,et al.  On the Separation of Signals from Neighboring Cells in Tetrode Recordings , 1997, NIPS.

[20]  J. Csicsvari,et al.  Intracellular features predicted by extracellular recordings in the hippocampus in vivo. , 2000, Journal of neurophysiology.

[21]  R. Quian Quiroga,et al.  Unsupervised Spike Detection and Sorting with Wavelets and Superparamagnetic Clustering , 2004, Neural Computation.

[22]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[23]  Michael Elad,et al.  Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit , 2008 .

[24]  Sacha Krstulovic,et al.  Mptk: Matching Pursuit Made Tractable , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[25]  M S Lewicki,et al.  A review of methods for spike sorting: the detection and classification of neural action potentials. , 1998, Network.

[26]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[27]  Yann LeCun,et al.  Convolutional Matching Pursuit and Dictionary Training , 2010, ArXiv.

[28]  Bradley Voytek,et al.  Nonsinusoidal Beta Oscillations Reflect Cortical Pathophysiology in Parkinson's Disease , 2017, The Journal of Neuroscience.

[29]  Kevin C. McGill,et al.  High-Resolution Alignment of Sampled Waveforms , 1984, IEEE Transactions on Biomedical Engineering.

[30]  G. Peyré,et al.  Sparse regularization on thin grids I: the Lasso , 2017 .

[31]  Andrew R. Teel,et al.  ESAIM: Control, Optimisation and Calculus of Variations , 2022 .

[32]  Eero P. Simoncelli,et al.  Recovery of Sparse Translation-Invariant Signals With Continuous Basis Pursuit , 2011, IEEE Transactions on Signal Processing.

[33]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[34]  Gongguo Tang,et al.  Sparse recovery over continuous dictionaries-just discretize , 2013, 2013 Asilomar Conference on Signals, Systems and Computers.

[35]  Lei Zhu,et al.  Faster STORM using compressed sensing , 2012, Nature Methods.

[36]  K. Bredies,et al.  Inverse problems in spaces of measures , 2013 .

[37]  R. Keys Cubic convolution interpolation for digital image processing , 1981 .