Bayesian compressive sensing and projection optimization

This paper introduces a new problem for which machine-learning tools may make an impact. The problem considered is termed "compressive sensing", in which a real signal of dimension N is measured accurately based on K << N real measurements. This is achieved under the assumption that the underlying signal has a sparse representation in some basis (e.g., wavelets). In this paper we demonstrate how techniques developed in machine learning, specifically sparse Bayesian regression and active learning, may be leveraged to this new problem. We also point out future research directions in compressive sensing of interest to the machine-learning community.

[1]  Christopher M. Bishop,et al.  Variational Relevance Vector Machines , 2000, UAI.

[2]  Bhaskar D. Rao,et al.  Perspectives on Sparse Bayesian Learning , 2003, NIPS.

[3]  J. Tropp,et al.  SIGNAL RECOVERY FROM PARTIAL INFORMATION VIA ORTHOGONAL MATCHING PURSUIT , 2005 .

[4]  David J. C. MacKay,et al.  Information-Based Objective Functions for Active Data Selection , 1992, Neural Computation.

[5]  David L. Donoho,et al.  Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit , 2006 .

[6]  Michael E. Tipping,et al.  Analysis of Sparse Bayesian Learning , 2001, NIPS.

[7]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[8]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[9]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[10]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[11]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[12]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[13]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[14]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[15]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[16]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[17]  Yaakov Tsaig,et al.  Extensions of compressed sensing , 2006, Signal Process..

[18]  Mário A. T. Figueiredo.  Adaptive Sparseness Using Jeffreys Prior , 2001, NIPS.

[19]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[20]  I. Miller.  Probability, Random Variables, and Stochastic Processes , 1966 .

[21]  Robert D. Nowak,et al.  Signal Reconstruction From Noisy Random Projections , 2006, IEEE Transactions on Information Theory.

[22]  Michael E. Tipping,et al.  Fast Marginal Likelihood Maximisation for Sparse Bayesian Models , 2003 .

[23]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[24]  William A. Pearlman,et al.  Efficient, low-complexity image coding with a set-partitioning embedded block coder , 2004, IEEE Transactions on Circuits and Systems for Video Technology.