Algorithms and bounds for sensing capacity and compressed sensing with applications to learning graphical models

We consider the problem of recovering sparse phenomena from projections of noisy data, a topic of interest in compressed sensing. We describe the problem in terms of sensing capacity, which we define as the supremum of the ratio of the number of signal dimensions that can be identified per projection. This notion quantifies minimum number of observations required to estimate a signal as a function of sensing channel, SNR, sensed environment(sparsity) as well as desired distortion up to which the sensed phenomena must be reconstructed. We first present bounds for two different sensing channels: (A) i.i.d. Gaussian observations (B) Correlated observations. We then extend the results derived for the correlated case to the problem of learning sparse graphical models. We then present convex programming methods for the different distortions for the correlated case. We then comment on the differences between the achievable bounds and the performance of convex programming methods.

[1]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[2]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[3]  Adam A. Margolin,et al.  Reverse engineering of regulatory networks in human B cells , 2005, Nature Genetics.

[4]  Joel A. Tropp,et al.  Recovery of short, complex linear combinations via /spl lscr//sub 1/ minimization , 2005, IEEE Transactions on Information Theory.

[5]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[6]  Steen Knudsen.  Reverse Engineering of Regulatory Networks , 2005 .

[7]  Robert D. Nowak,et al.  Signal Reconstruction From Noisy Random Projections , 2006, IEEE Transactions on Information Theory.

[8]  D. Donoho.  For most large underdetermined systems of equations, the minimal 𝓁1‐norm near‐solution approximates the sparsest near‐solution , 2006 .

[9]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[10]  Richard G. Baraniuk,et al.  Sparse Signal Detection from Incoherent Projections , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[11]  Martin J. Wainwright,et al.  Sharp thresholds for high-dimensional and noisy recovery of sparsity , 2006, ArXiv.

[12]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[13]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[14]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[15]  Venkatesh Saligrama,et al.  On sensing capacity of sensor networks for a class of linear observation models , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[16]  V. Saligrama,et al.  Information theoretic bounds to sensing capacity of sensor networks under fixed SNR , 2007, 2007 IEEE Information Theory Workshop.

[17]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .