Bayesian Estimation for Continuous-Time Sparse Stochastic Processes

We consider continuous-time sparse stochastic processes from which we have only a finite number of noisy/noiseless samples. Our goal is to estimate the noiseless samples (denoising) and the signal in-between (interpolation problem). By relying on tools from the theory of splines, we derive the joint a priori distribution of the samples and show how this probability density function can be factorized. The factorization enables us to tractably implement the maximum a posteriori and minimum mean-square error (MMSE) criteria as two statistical approaches for estimating the unknowns. We compare the derived statistical methods with well-known techniques for the recovery of sparse signals, such as the ℓ1 norm and Log (ℓ1-ℓ0 relaxation) regularization methods. The simulation results show that, under certain conditions, the performance of the regularization techniques can be very close to that of the MMSE estimator.

[1]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[2]  Thierry Blu,et al.  Generalized smoothing splines and the optimal discretization of the Wiener filter , 2005, IEEE Transactions on Signal Processing.

[3]  Michael Unser,et al.  A unified formulation of Gaussian vs. sparse stochastic processes - Part I: Continuous-domain theory , 2011, ArXiv.

[4]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[5]  Thierry Blu,et al.  Sampling signals with finite rate of innovation , 2002, IEEE Trans. Signal Process..

[6]  Eero P. Simoncelli,et al.  Learning to be Bayesian without Supervision , 2006, NIPS.

[7]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[8]  佐藤 健一 Lévy processes and infinitely divisible distributions , 2013 .

[9]  Li Ping,et al.  The Factor Graph Approach to Model-Based Signal Processing , 2007, Proceedings of the IEEE.

[10]  Michael Unser,et al.  MMSE denoising of sparse Lévy processes via message passing , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[11]  Thierry Blu,et al.  Self-Similarity: Part II—Optimal Estimation of Fractal Processes , 2007, IEEE Transactions on Signal Processing.

[12]  P. Carr,et al.  The Variance Gamma Process and Option Pricing , 1998 .

[13]  Michael Unser,et al.  Compressibility of Deterministic and Random Infinite Sequences , 2011, IEEE Transactions on Signal Processing.

[14]  Thierry Blu,et al.  Cardinal exponential splines: part I - theory and filtering algorithms , 2005, IEEE Transactions on Signal Processing.

[15]  Michael Elad,et al.  Submitted to Ieee Transactions on Image Processing Image Decomposition via the Combination of Sparse Representations and a Variational Approach , 2022 .

[16]  G. Casella,et al.  The Bayesian Lasso , 2008 .

[17]  F. Steutel,et al.  Infinite Divisibility of Probability Distributions on the Real Line , 2003 .

[18]  Michael Unser,et al.  A unified formulation of Gaussian vs. sparse stochastic processes - Part II: Discrete-domain theory , 2011, ArXiv.

[19]  Thierry Blu,et al.  Self-Similarity: Part I—Splines and Operators , 2007, IEEE Transactions on Signal Processing.

[20]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[21]  Michael Unser,et al.  MMSE Estimation of Sparse Lévy Processes , 2013, IEEE Transactions on Signal Processing.

[22]  C. Stein Estimation of the Mean of a Multivariate Normal Distribution , 1981 .

[23]  Thierry Blu,et al.  A New SURE Approach to Image Denoising: Interscale Orthonormal Wavelet Thresholding , 2007, IEEE Transactions on Image Processing.

[24]  Valentin Simeonov,et al.  École polytechnique fédérale de Lausanne (EPFL) , 2018, The Grants Register 2019.

[25]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[26]  Rémi Gribonval,et al.  Should Penalized Least Squares Regression be Interpreted as Maximum A Posteriori Estimation? , 2011, IEEE Transactions on Signal Processing.

[27]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[28]  Amiel Feinstein,et al.  Applications of harmonic analysis , 1964 .

[29]  Volkan Cevher,et al.  Learning with Compressible Priors , 2009, NIPS.

[30]  Michael Unser,et al.  Stochastic Models for Sparse and Piecewise-Smooth Signals , 2011, IEEE Transactions on Signal Processing.

[31]  Volkan Cevher,et al.  Compressible Distributions for High-Dimensional Statistics , 2011, IEEE Transactions on Information Theory.

[32]  D. Applebaum Stable non-Gaussian random processes , 1995, The Mathematical Gazette.

[33]  Bhaskar D. Rao,et al.  Sparse Bayesian learning for basis selection , 2004, IEEE Transactions on Signal Processing.