Lower bounds on the rate-distortion function of LDGM codes

We analyze the performance of low-density generator matrix (LDGM) codes for lossy source coding. We first develop a generic technique for deriving lower bounds on the effective rate-distortion functions of binary linear codes. This result provides a source coding analog of a classical result due to Gallager for channel coding over the binary symmetric channel. We illustrate this method for the ensemble of check-regular low- density generator matrix (LDGM) codes by deriving an explicit lower bound on its rate-distortion performance as a function of the check degree.

[1]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[2]  Michael Horstein,et al.  Review of 'Low-Density Parity-Check Codes' (Gallager, R. G.; 1963) , 1964, IEEE Transactions on Information Theory.

[3]  Andrew J. Viterbi,et al.  Trellis Encoding of memoryless discrete-time sources with a fidelity criterion , 1974, IEEE Trans. Inf. Theory.

[4]  Michael W. Marcellin,et al.  Trellis coded quantization of memoryless and Gauss-Markov sources , 1990, IEEE Trans. Commun..

[5]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[6]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[7]  Riccardo Zecchina,et al.  Alternative solutions to diluted p-spin models and XORSAT problems , 2002, ArXiv.

[8]  M. Mézard,et al.  Analytic and Algorithmic Solution of Random Satisfiability Problems , 2002, Science.

[9]  Olivier Dubois,et al.  The 3-XORSAT threshold , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[10]  Hirosuke Yamamoto,et al.  A coding theorem for lossy data compression by LDPC codes , 2003, IEEE Trans. Inf. Theory.

[11]  R. Monasson,et al.  Rigorous decimation-based construction of ground pure states for spin-glass models on random lattices. , 2002, Physical review letters.

[12]  Nadia Creignou,et al.  Approximating The Satisfiability Threshold For Random K-Xor-Formulas , 2003, Comb. Probab. Comput..

[13]  M. Mézard,et al.  Two Solutions to Diluted p-Spin Models and XORSAT Problems , 2003 .

[14]  T. Murayama.  Thouless-Anderson-Palmer approach for lossy compression. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[15]  Emin Martinian,et al.  Iterative Quantization Using Codes On Graphs , 2004, ArXiv.

[16]  Noga Alon,et al.  The Probabilistic Method, Second Edition , 2004 .

[17]  Martin J. Wainwright,et al.  Lossy source encoding via message-passing and decimation over generalized codewords of LDGM codes , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[18]  Eli Upfal,et al.  Probability and Computing: Randomized Algorithms and Probabilistic Analysis , 2005 .

[19]  Ali Esmaili,et al.  Probability and Random Processes , 2005, Technometrics.

[20]  Marc Mézard,et al.  The theoretical capacity of the Parity Source Coder , 2005, ArXiv.

[21]  Martin J. Wainwright,et al.  Low-density constructions can achieve the Wyner-Ziv and Gelfand-Pinsker bounds , 2006, 2006 IEEE International Symposium on Information Theory.

[22]  Martin J. Wainwright,et al.  Analysis of LDGM and compound codes for lossy compression and binning , 2006, ArXiv.

[23]  R. Zecchina,et al.  Message-Passing Algorithms for Non-Linear Nodes and Data Compression , 2005, Complexus.

[24]  Martin J. Wainwright,et al.  Low density codes achieve the rate-distortion bound , 2006, Data Compression Conference (DCC'06).

[25]  Jessica J. Fridrich,et al.  Binary quantization using Belief Propagation with decimation over factor graphs of LDGM codes , 2007, ArXiv.

[26]  Rüdiger L. Urbanke,et al.  Lower bounds on the rate-distortion function of individual LDGM codes , 2008, 2008 5th International Symposium on Turbo Codes and Related Topics.

[27]  Sergio Verdú,et al.  Nonlinear Sparse-Graph Codes for Lossy Compression , 2009, IEEE Transactions on Information Theory.