Continuous-Domain Formulation of Inverse Problems for Composite Sparse-Plus-Smooth Signals

We present a novel framework for the reconstruction of 1D composite signals assumed to be a mixture of two additive components, one sparse and the other smooth, given a finite number of linear measurements. We formulate the reconstruction problem as a continuous-domain regularized inverse problem with multiple penalties. We prove that these penalties induce reconstructed signals that indeed take the desired form of the sum of a sparse and a smooth component. We then discretize this problem using Riesz bases, which yields a discrete problem that can be solved by standard algorithms. Our discretization is exact in the sense that we are solving the continuous-domain problem over the search space specified by our bases without any discretization error. We propose a complete algorithmic pipeline and demonstrate its feasibility on simulated data.

[1]  Emmanuel J. Candès,et al.  Towards a Mathematical Theory of Super‐resolution , 2012, ArXiv.

[2]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[3]  Ali Gholami,et al.  A balanced combination of Tikhonov and total variation regularizations for reconstruction of piecewise-smooth signals , 2013, Signal Process..

[4]  Zewen Wang,et al.  Multi-parameter Tikhonov regularization and model function approach to the damped Morozov principle for choosing regularization parameters , 2012, J. Comput. Appl. Math..

[5]  E. Miller,et al.  Efficient determination of multiple regularization parameters in a generalized L-curve framework , 2002 .

[6]  lexander,et al.  THE GENERALIZED SIMPLEX METHOD FOR MINIMIZING A LINEAR FORM UNDER LINEAR INEQUALITY RESTRAINTS , 2012 .

[7]  Michael Unser,et al.  Hybrid-Spline Dictionaries for Continuous-Domain Inverse Problems , 2019, IEEE Transactions on Signal Processing.

[8]  Stanley Osher,et al.  Image Denoising and Decomposition with Total Variation Minimization and Oscillatory Functions , 2004, Journal of Mathematical Imaging and Vision.

[9]  Paul Escande,et al.  Learning Low-Dimensional Models of Microscopes , 2020, IEEE Transactions on Computational Imaging.

[10]  Michael Unser Cardinal exponential splines: part II - think analog, act digital , 2005, IEEE Transactions on Signal Processing.

[11]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[12]  Abhishake Rastogi,et al.  Multi-penalty regularization in learning theory , 2016, J. Complex..

[13]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[14]  Thierry Blu,et al.  Generalized smoothing splines and the optimal discretization of the Wiener filter , 2005, IEEE Transactions on Signal Processing.

[15]  E.J. Candes Compressive Sampling , 2022 .

[16]  Michel Defrise,et al.  Inverse imaging with mixed penalties , 2004 .

[17]  Joseph W. Jerome,et al.  Spline solutions to L1 extremal problems in one and several variables , 1975 .

[18]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[19]  Timo Klock,et al.  Adaptive multi-penalty regularization based on a generalized Lasso path , 2017, Applied and Computational Harmonic Analysis.

[20]  Michael Unser,et al.  Generating Sparse Stochastic Processes Using Matched Splines , 2020, IEEE Transactions on Signal Processing.

[21]  A. Amini,et al.  A universal formula for generalized cardinal B-splines , 2016, Applied and Computational Harmonic Analysis.

[22]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[23]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[24]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[25]  V. Naumova,et al.  Minimization of multi-penalty functionals by alternating iterative thresholding and optimal parameter choices , 2014, 1403.6718.

[26]  Ben Adcock,et al.  Generalized Sampling and Infinite-Dimensional Compressed Sensing , 2016, Found. Comput. Math..

[27]  L. Schwartz Théorie des distributions , 1966 .

[28]  Shuai Lu,et al.  Multi-parameter regularization and its numerical realization , 2011, Numerische Mathematik.

[29]  Zhongying,et al.  MULTI-PARAMETER TIKHONOV REGULARIZATION FOR LINEAR ILL-POSED OPERATOR EQUATIONS , 2008 .

[30]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[31]  Michael J. Black,et al.  Fields of Experts: a framework for learning image priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[32]  Michael Unser,et al.  Computation of "Best" Interpolants in the Lp Sense , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[33]  Michael Unser,et al.  Representer Theorems for Sparsity-Promoting $\ell _{1}$ Regularization , 2016, IEEE Transactions on Information Theory.

[34]  K. Bredies,et al.  Sparsity of solutions for variational inverse problems with finite-dimensional data , 2018, Calculus of Variations and Partial Differential Equations.

[35]  Michael Unser,et al.  Periodic Splines and Gaussian Processes for the Resolution of Linear Inverse Problems , 2018, IEEE Transactions on Signal Processing.

[36]  C. R. Deboor,et al.  A practical guide to splines , 1978 .

[37]  P. Weiss,et al.  Exact solutions of infinite dimensional total-variation regularized problems , 2017, Information and Inference: A Journal of the IMA.

[38]  R. D. Grigorieff,et al.  On Cardinal Spline Interpolation , 2013, Comput. Methods Appl. Math..

[39]  Antonin Chambolle,et al.  On Representer Theorems and Convex Regularization , 2018, SIAM J. Optim..

[40]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[41]  Yves Meyer,et al.  Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures , 2001 .

[42]  Michael Unser,et al.  A representer theorem for deep neural networks , 2018, J. Mach. Learn. Res..

[43]  M. Unser,et al.  Native Banach spaces for splines and variational inverse problems , 2019, 1904.10818.

[44]  Michael Unser,et al.  Splines Are Universal Solutions of Linear Inverse Problems with Generalized TV Regularization , 2016, SIAM Rev..

[45]  I. Daubechies,et al.  Sparsity-enforcing regularisation and ISTA revisited , 2016 .

[46]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[47]  Stanley Osher,et al.  Modeling Textures with Total Variation Minimization and Oscillating Patterns in Image Processing , 2003, J. Sci. Comput..

[48]  Akram Aldroubi,et al.  B-SPLINE SIGNAL PROCESSING: PART I-THEORY , 1993 .

[49]  A Tikhonov,et al.  Solution of Incorrectly Formulated Problems and the Regularization Method , 1963 .

[50]  Thierry Blu,et al.  Cardinal exponential splines: part I - theory and filtering algorithms , 2005, IEEE Transactions on Signal Processing.

[51]  Michael Unser,et al.  Continuous-Domain Solutions of Linear Inverse Problems With Tikhonov Versus Generalized TV Regularization , 2018, IEEE Transactions on Signal Processing.

[52]  Matthieu Simeoni,et al.  TV-based reconstruction of periodic functions , 2020, Inverse Problems.

[53]  Michael Unser,et al.  Multi-Kernel Regression with Sparsity Constraint , 2018 .

[54]  G. Wahba Spline models for observational data , 1990 .

[55]  Michael Unser,et al.  Splines: a perfect fit for signal and image processing , 1999, IEEE Signal Process. Mag..

[56]  Michael Unser,et al.  B-Spline-Based Exact Discretization of Continuous-Domain Inverse Problems With Generalized TV Regularization , 2019, IEEE Transactions on Information Theory.