Sparsity in Bayesian Signal Estimation

In this chapter, we describe different methods to estimate an unknown signal from its linear measurements. We focus on the underdetermined case where the number of measurements is less than the dimension of the unknown signal. We introduce the concept of signal sparsity and describe how it could be used as prior information for either regularized least squares or Bayesian signal estimation. We discuss compressed sensing and sparse signal representation as examples where these sparse signal estimation methods could be applied.

[1]  T. Moon,et al.  Mathematical Methods and Algorithms for Signal Processing , 1999 .

[2]  Ronald R. Coifman,et al.  Entropy-based algorithms for best basis selection , 1992, IEEE Trans. Inf. Theory.

[3]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[4]  Stéphane Mallat,et al.  A Wavelet Tour of Signal Processing - The Sparse Way, 3rd Edition , 2008 .

[5]  M. Salman Asif,et al.  Dynamic compressive sensing: sparse recovery algorithms for streaming signals and video , 2013 .

[6]  Lotfi A. Zadeh,et al.  General System Theory , 1962 .

[7]  Tomaso Poggio,et al.  Computational vision and regularization theory , 1985, Nature.

[8]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[9]  Claude E. Shannon,et al.  The Mathematical Theory of Communication , 1950 .

[10]  Paul Fieguth,et al.  Statistical Image Processing and Multidimensional Modeling , 2010 .

[11]  Bhaskar D. Rao,et al.  An affine scaling methodology for best basis selection , 1999, IEEE Trans. Signal Process..

[12]  J. Mendel.  Lessons in Estimation Theory for Signal Processing, Communications, and Control , 1995 .

[13]  H. Jonas,et al.  General system theory; a new approach to unity of science. 4. Comment on general system theory. , 1951, Human biology.

[14]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[15]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[16]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[17]  Simon J. D. Prince,et al.  Computer Vision: Index , 2012 .

[18]  Daniel D. Lee,et al.  Bayesian L1-Norm Sparse Learning , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[19]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[20]  H. Sorenson.  Least-squares estimation: from Gauss to Kalman , 1970, IEEE Spectrum.

[21]  T. Blumensath,et al.  Theory and Applications , 2011 .

[22]  P. Bromiley.  Products and Convolutions of Gaussian Probability Density Functions , 2013 .

[23]  K. Shanmugan,et al.  Random Signals: Detection, Estimation and Data Analysis , 1988 .

[24]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[25]  Ke Huang,et al.  Sparse Representation for Signal Classification , 2006, NIPS.

[26]  Karel J. Keesman,et al.  System Identification: An Introduction , 2011 .

[27]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[28]  P. Halmos.  Finite-Dimensional Vector Spaces , 1960 .

[29]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[30]  Albert Tarantola,et al.  Inverse problem theory - and methods for model parameter estimation , 2004 .

[31]  Chi-Tsong Chen,et al.  Linear System Theory and Design , 1995 .

[32]  S. Mallat.  VI – Wavelet zoom , 1999 .

[33]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[34]  Peter E. Wellstead.  Non-parametric methods of system identification , 1981, Autom..

[35]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[36]  G. Wahba,et al.  Some New Mathematical Methods for Variational Objective Analysis Using Splines and Cross Validation , 1980 .

[37]  Ingrid Daubechies,et al.  Time-frequency localization operators: A geometric phase space approach , 1988, IEEE Trans. Inf. Theory.

[38]  Lennart Ljung,et al.  Perspectives on system identification , 2010, Annu. Rev. Control..

[39]  Simon J. D. Prince,et al.  Computer Vision: Models, Learning, and Inference , 2012 .