An Expectation Propagation Perspective on Approximate Message Passing

An alternative derivation for the well-known approximate message passing (AMP) algorithm proposed by Donoho is presented in this letter. Compared with the original derivation, which exploits central limit theorem and Taylor expansion to simplify belief propagation (BP), our derivation resorts to expectation propagation (EP) and the neglect of high-order terms in large system limit. This alternative derivation leads to a different yet provably equivalent form of message passing, which explicitly establishes the intrinsic connection between AMP and EP, thereby offering some new insights in the understanding and improvement of AMP.

[1]  Jianhua Lu,et al.  Low-Complexity Iterative Detection for Large-Scale Multiuser MIMO-OFDM Systems Using Approximate Message Passing , 2014, IEEE Journal of Selected Topics in Signal Processing.

[2]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[3]  Tom Minka,et al.  A family of algorithms for approximate Bayesian inference , 2001 .

[4]  Andrea Montanari,et al.  The dynamics of message passing on dense graphs, with applications to compressed sensing , 2010, ISIT.

[5]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[6]  W. Wiegerinck,et al.  Approximate inference techniques with expectation constraints , 2005 .

[7]  Michael Unser,et al.  Approximate Message Passing With Consistent Parameter Estimation and Applications to Sparse Learning , 2012, IEEE Transactions on Information Theory.

[8]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[9]  Gitta Kutyniok,et al.  1 . 2 Sparsity : A Reasonable Assumption ? , 2012 .

[10]  Florent Krzakala,et al.  Variational free energies for compressed sensing , 2014, 2014 IEEE International Symposium on Information Theory.

[11]  Sundeep Rangan,et al.  Generalized approximate message passing for estimation with random linear mixing , 2010, 2011 IEEE International Symposium on Information Theory Proceedings.

[12]  Andrea Montanari,et al.  Graphical Models Concepts in Compressed Sensing , 2010, Compressed Sensing.

[13]  Andrea Montanari,et al.  Message passing algorithms for compressed sensing: I. motivation and construction , 2009, 2010 IEEE Information Theory Workshop on Information Theory (ITW 2010, Cairo).

[14]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[15]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[16]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[17]  Ole Winther,et al.  S-AMP: Approximate message passing for general matrix ensembles , 2014, 2014 IEEE Information Theory Workshop (ITW 2014).

[18]  Thomas P. Minka,et al.  Divergence measures and message passing , 2005 .

[19]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[20]  Philip Schniter,et al.  A Message-Passing Receiver for BICM-OFDM Over Unknown Clustered-Sparse Channels , 2011, IEEE Journal of Selected Topics in Signal Processing.

[21]  X. Jin.  Factor graphs and the Sum-Product Algorithm , 2002 .