A low density lattice decoder via non-parametric belief propagation

The recent work of Sommer, Feder and Shalvi presented a new family of codes called low density lattice codes (LDLC) that can be decoded efficiently and approach the capacity of the AWGN channel. A linear time iterative decoding scheme which is based on a message-passing formulation on a factor graph is given. In the current work we report our theoretical findings regarding the relation between the LDLC decoder and belief propagation. We show that the LDLC decoder is an instance of non-parametric belief propagation and further connect it to the Gaussian belief propagation algorithm. Our new results enable borrowing knowledge from the non-parametric and Gaussian belief propagation domains into the LDLC domain. Specifically, we give more general convergence conditions for convergence of the LDLC decoder (under the same assumptions of the original LDLC convergence analysis). We discuss how to extend the LDLC decoder from Latin square to full rank, non-square matrices. We propose an efficient construction of sparse generator matrix and its matching decoder. We report preliminary experimental results which show our decoder has comparable symbol to error rate compared to the original LDLC decoder.

[1]  Dmitry M. Malioutov,et al.  Walk-Sums and Belief Propagation in Gaussian Graphical Models , 2006, J. Mach. Learn. Res..

[2]  John W. Fisher,et al.  Loopy Belief Propagation: Convergence and Effects of Message Errors , 2005, J. Mach. Learn. Res..

[3]  Meir Feder,et al.  Low Density Lattice Codes , 2006, ISIT.

[4]  William T. Freeman,et al.  Nonparametric belief propagation , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[5]  Richard Baraniuk,et al.  Compressed Sensing Reconstruction via Belief Propagation , 2006 .

[6]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[7]  A. Willsky,et al.  Particle filtering under communications constraints , 2005, IEEE/SP 13th Workshop on Statistical Signal Processing, 2005.

[8]  Daniel Rudoy,et al.  Multi-Scale MCMC Methods for Sampling from Products of Gaussian Mixtures , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[9]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[10]  Richard G. Baraniuk,et al.  Bayesian Compressive Sensing Via Belief Propagation , 2008, IEEE Transactions on Signal Processing.

[11]  X. Jin Factor graphs and the Sum-Product Algorithm , 2002 .

[12]  William T. Freeman,et al.  Correctness of Belief Propagation in Gaussian Graphical Models of Arbitrary Topology , 1999, Neural Computation.

[13]  Paul H. Siegel,et al.  Gaussian belief propagation solver for systems of linear equations , 2008, 2008 IEEE International Symposium on Information Theory.

[14]  Gregory Poltyrev,et al.  On coding without restrictions for the AWGN channel , 1993, IEEE Trans. Inf. Theory.

[15]  Meir Feder,et al.  Efficient parametric decoder of low density lattice codes , 2009, 2009 IEEE International Symposium on Information Theory.

[16]  William T. Freeman,et al.  Efficient Multiscale Sampling from Products of Gaussian Mixtures , 2003, NIPS.

[17]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[18]  Danny Bickson,et al.  Gaussian Belief Propagation: Theory and Aplication , 2008, 0811.2518.

[19]  Jung-Fu Cheng,et al.  Turbo Decoding as an Instance of Pearl's "Belief Propagation" Algorithm , 1998, IEEE J. Sel. Areas Commun..

[20]  John W. Fisher,et al.  Nonparametric belief propagation for self-localization of sensor networks , 2005, IEEE Journal on Selected Areas in Communications.

[21]  Justin Dauwels,et al.  Message-passing decoding of lattices using Gaussian mixtures , 2008, 2008 IEEE International Symposium on Information Theory.

[22]  Michael I. Mandel,et al.  Visual Hand Tracking Using Nonparametric Belief Propagation , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[23]  Paul H. Siegel,et al.  Gaussian belief propagation based multiuser detection , 2008, 2008 IEEE International Symposium on Information Theory.

[24]  Danny Dolev,et al.  Fixing convergence of Gaussian belief propagation , 2009, 2009 IEEE International Symposium on Information Theory.

[25]  A. Doucet,et al.  Sequential auxiliary particle belief propagation , 2005, 2005 7th International Conference on Information Fusion.

[26]  Jiří Zelinka,et al.  Kernel Density Estimation Toolbox for Matlab , 2011 .

[27]  Brian M. Kurkoski,et al.  Single-Gaussian messages and noise thresholds for decoding low-density lattice codes , 2009, 2009 IEEE International Symposium on Information Theory.

[28]  Meir Feder,et al.  Low-Density Lattice Codes , 2007, IEEE Transactions on Information Theory.