Stabilization Techniques for Iterative Algorithms in Compressed Sensing

Algorithms for signal recovery in compressed sensing (CS) are often improved by stabilization techniques, such as damping, or the less widely known so-called fractional approach, which is based on the expectation propagation (EP) framework. These procedures are used to increase the steady-state performance, i.e., the performance after convergence, or assure convergence, when this is otherwise not possible. In this paper, we give a thorough introduction and interpretation of several stabilization approaches. The effects of the stabilization procedures are examined and compared via numerical simulations and we show that a combination of several procedures can be beneficial for the performance of the algorithm.

[1]  Tom Heskes,et al.  Fractional Belief Propagation , 2002, NIPS.

[2]  Ole Winther,et al.  Expectation Consistent Approximate Inference , 2005, J. Mach. Learn. Res..

[3]  Yingzhen Li.  A Unifying Approximate Inference Framework from Variational Free Energy Relaxation , 2016 .

[4]  Sundeep Rangan,et al.  Vector approximate message passing , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[5]  Thomas P. Minka,et al.  Divergence measures and message passing , 2005 .

[6]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[7]  Robert F. H. Fischer,et al.  On the Invariance of Recovery Algorithms for Compressed Sensing based on Expectation-Consistent Approximate Inference , 2020, WSA.

[8]  Mikko Vehkaperä,et al.  Signal recovery using expectation consistent approximation for linear observations , 2014, 2014 IEEE International Symposium on Information Theory.

[9]  Sundeep Rangan,et al.  Adaptive damping and mean removal for the generalized approximate message passing algorithm , 2014, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  Matthias W. Seeger,et al.  Compressed sensing and Bayesian experimental design , 2008, ICML '08.

[11]  Florian Steinke,et al.  Bayesian Inference and Optimal Design in the Sparse Linear Model , 2007, AISTATS.

[12]  L. Brown.  Fundamentals of statistical exponential families: with applications in statistical decision theory , 1986 .

[13]  M. Seeger.  Expectation Propagation for Exponential Families , 2005 .

[14]  Sundeep Rangan,et al.  Expectation consistent approximate inference: Generalizations and convergence , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[15]  Ole Winther,et al.  S-AMP: Approximate message passing for general matrix ensembles , 2014, 2014 IEEE Information Theory Workshop (ITW 2014).

[16]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[17]  Robert F. H. Fischer,et al.  VAMP with Vector-Valued Diagonalization , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[18]  Tom Minka,et al.  Expectation Propagation for approximate Bayesian inference , 2001, UAI.

[19]  Robert F. H. Fischer,et al.  Algorithms for the Iterative Estimation of Discrete-Valued Sparse Vectors , 2016, ArXiv.

[20]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[21]  Robert F. H. Fischer,et al.  VAMP with Individual Variances and Sequential Processing for Compressed Sensing , 2021, 2021 29th European Signal Processing Conference (EUSIPCO).

[22]  T. Minka.  Power EP , 2004 .