An Approximate Message Passing Framework for Side Information

Approximate message passing (AMP) methods have gained recent traction in sparse signal recovery. Additional information about the signal, or side information (SI), is commonly available and can aid in efficient signal recovery. This paper presents an AMP-based framework that exploits SI and can be readily implemented in various settings for which the SI results in separable distributions. To illustrate the simplicity and applicability of our approach, this framework is applied to a Bernoulli–Gaussian (BG) model and a time-varying birth-death-drift (BDD) signal model, motivated by applications in channel estimation. We develop a suite of algorithms, called AMP-SI, and derive denoisers for the BDD and BG models. Numerical evidence demonstrating the advantages of our approach are presented alongside empirical evidence of the accuracy of a proposed state evolution.

[1]  R. Gallager.  Information Theory and Reliable Communication , 1968 .

[2]  Michael Unser,et al.  Sparse Image Deconvolution with Message Passing , 2013 .

[3]  Yonina C. Eldar,et al.  Compressed sensing for longitudinal MRI: An adaptive-weighted approach. , 2014, Medical physics.

[4]  Henry Arguello,et al.  Code aperture optimization for spectrally agile compressive imaging. , 2011, Journal of the Optical Society of America. A, Optics, image science, and vision.

[5]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[6]  Miguel R. D. Rodrigues,et al.  On the design of linear projections for compressive sensing with side information , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[7]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[8]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[9]  Dror Baron,et al.  An Analysis of State Evolution for Approximate Message Passing with Side Information , 2019, 2019 IEEE International Symposium on Information Theory (ISIT).

[10]  Miguel R. D. Rodrigues,et al.  Compressed Sensing With Prior Information: Strategies, Geometry, and Bounds , 2017, IEEE Transactions on Information Theory.

[11]  Michael Unser,et al.  Approximate Message Passing With Consistent Parameter Estimation and Applications to Sparse Learning , 2012, IEEE Transactions on Information Theory.

[12]  Nicolas Macris,et al.  Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation , 2017, IEEE Transactions on Information Theory.

[13]  Galen Reeves,et al.  The replica-symmetric prediction for compressed sensing with Gaussian matrices is exact , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[14]  Yanting Ma,et al.  Compressive Imaging via Approximate Message Passing With Image Denoising , 2014, IEEE Transactions on Signal Processing.

[15]  Ramji Venkataramanan,et al.  Finite Sample Analysis of Approximate Message Passing Algorithms , 2016, IEEE Transactions on Information Theory.

[16]  Richard G. Baraniuk,et al.  A new compressive imaging camera architecture using optical-domain compression , 2006, Electronic Imaging.

[17]  Rayan Saab,et al.  Weighted ℓ1-Minimization for Sparse Recovery under Arbitrary Prior Information , 2016, ArXiv.

[18]  Ramji Venkataramanan,et al.  Finite-sample analysis of Approximate Message Passing , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[19]  Hassan Mansour,et al.  Recovery Analysis for Weighted ℓ1-Minimization Using a Null Space Property , 2014, ArXiv.

[20]  Ahmad Beirami,et al.  Optimal Trade-offs in Multi-Processor Approximate Message Passing , 2016, 1601.03790.

[21]  Florent Krzakala,et al.  Probabilistic reconstruction in compressed sensing: algorithms, phase diagrams, and threshold achieving matrices , 2012, ArXiv.

[22]  Florent Krzakala,et al.  Streaming Bayesian inference: Theoretical limits and mini-batch approximate message-passing , 2017, 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[23]  Andrea Montanari,et al.  The dynamics of message passing on dense graphs, with applications to compressed sensing , 2010, ISIT.

[24]  Deanna Needell,et al.  Conditional approximate message passing with side information , 2017, 2017 51st Asilomar Conference on Signals, Systems, and Computers.

[25]  Sundeep Rangan,et al.  Plug in estimation in high dimensional linear inverse problems a rigorous analysis , 2018, NeurIPS.

[26]  André Kaup,et al.  Sparse signal recovery with multiple prior information: Algorithm and measurement bounds , 2018, Signal Process..

[27]  Volkan Cevher,et al.  Adaptive-Rate Reconstruction of Time-Varying Signals With Application in Compressive Foreground Extraction , 2016, IEEE Transactions on Signal Processing.

[28]  Ahmad Beirami,et al.  Performance trade-offs in multi-processor approximate message passing , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[29]  P. Mohana Shankar,et al.  Fading and Shadowing in Wireless Systems , 2011 .

[30]  Wei Lu,et al.  Modified-CS: Modifying compressive sensing for problems with partially known support , 2009, 2009 IEEE International Symposium on Information Theory.

[31]  A. Robert Calderbank,et al.  Classification and Reconstruction of High-Dimensional Signals From Low-Dimensional Features in the Presence of Side Information , 2014, IEEE Transactions on Information Theory.

[32]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[33]  Sundeep Rangan,et al.  Generalized approximate message passing for estimation with random linear mixing , 2010, 2011 IEEE International Symposium on Information Theory Proceedings.

[34]  Xing Wang,et al.  Approximate message passing-based compressed sensing reconstruction with generalized elastic net prior , 2013, Signal Process. Image Commun..

[35]  André Kaup,et al.  Performance Bounds for Sparse Signal Reconstruction with Multiple Side Information , 2016, ArXiv.

[36]  Sergio Verdú,et al.  Randomly spread CDMA: asymptotics via statistical physics , 2005, IEEE Transactions on Information Theory.

[37]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[38]  Sundeep Rangan,et al.  Vector approximate message passing , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[39]  Soren Forchhammer,et al.  Measurement Bounds for Sparse Signal Recovery With Multiple Side Information , 2017 .

[40]  Desmond P. Taylor,et al.  A Statistical Model for Indoor Multipath Propagation , 2007 .

[41]  Yanting Ma,et al.  Approximate Message Passing Algorithm With Universal Denoising and Gaussian Mixture Learning , 2015, IEEE Transactions on Signal Processing.

[42]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[43]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[44]  Florent Krzakala,et al.  Approximate message-passing with spatially coupled structured operators, with applications to compressed sensing and sparse superposition codes , 2013, 1312.1740.

[45]  Ralf R. Müller,et al.  Iterative multiuser joint decoding: optimal power allocation and low-complexity implementation , 2004, IEEE Transactions on Information Theory.

[46]  Toshiyuki Tanaka,et al.  A statistical-mechanics approach to large-system analysis of CDMA multiuser detectors , 2002, IEEE Trans. Inf. Theory.

[47]  Sundeep Rangan,et al.  On the convergence of approximate message passing with arbitrary matrices , 2014, 2014 IEEE International Symposium on Information Theory.

[48]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[49]  Andrea Montanari,et al.  The dynamics of message passing on dense graphs, with applications to compressed sensing , 2010, 2010 IEEE International Symposium on Information Theory.

[50]  Sundeep Rangan,et al.  Inference for Generalized Linear Models via Alternating Directions and Bethe Free Energy Minimization , 2015, IEEE Transactions on Information Theory.

[51]  YuanXin,et al.  Classification and Reconstruction of High-Dimensional Signals From Low-Dimensional Features in the Presence of Side Information , 2016 .

[52]  Philip Schniter,et al.  Dynamic Compressive Sensing of Time-Varying Signals Via Approximate Message Passing , 2012, IEEE Transactions on Signal Processing.

[53]  Jie Tang,et al.  Prior image constrained compressed sensing (PICCS): a method to accurately reconstruct dynamic CT images from highly undersampled projection data sets. , 2008, Medical physics.

[54]  Andrea Montanari,et al.  State Evolution for Approximate Message Passing with Non-Separable Functions , 2017, Information and Inference: A Journal of the IMA.

[55]  Theodore S. Rappaport,et al.  Rapid Fading Due to Human Blockage in Pedestrian Crowds at 5G Millimeter-Wave Frequencies , 2017, GLOBECOM 2017 - 2017 IEEE Global Communications Conference.

[56]  R. Gribonval,et al.  Exact Recovery Conditions for Sparse Representations With Partial Support Information , 2013, IEEE Transactions on Information Theory.