Analysis of LDGM and compound codes for lossy compression and binning

Recent work has suggested that low-density gener- ator matrix (LDGM) codes are likely to be effective for lossy source coding problems. We derive rigorous upper bounds on the effective rate-distortion function of LDGM codes for the binary symmetric source, showing that they quickly approach the rate-distortion function as the degree increases. We also compare and contrast the standard LDGM construction with a compound LDPC/LDGM construction introduced in our previous work, which provably saturates the rate-distortion bound with finite degrees. Moreover, this compound construction can be used to generate nested codes that are simultaneously good as source and channel codes, and are hence well-suited to source/channel coding with side information. The sparse and high-girth graphical structure of our constructions render them well-suited to message-passing encoding. For channel coding problems, codes based on graphical con- structions, including turbo codes and low-density parity c heck (LDPC) codes, are widely used and well understood (16). However, many communication problems involve aspects of quantization, or quantization in conjunction with channel coding. Well-known examples include lossy data compression, source coding with side information (the Wyner-Ziv problem), and channel coding with side information (the Gelfand-Pinsker problem). For such communication problems involving quanti- zation, the use of sparse graphical codes and message-passing

[1]  Emin Martinian,et al.  Iterative Quantization Using Codes On Graphs , 2004, ArXiv.

[2]  Olivier Dubois,et al.  The 3-XORSAT threshold , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[3]  Marc Mézard,et al.  The theoretical capacity of the Parity Source Coder , 2005, ArXiv.

[4]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[5]  M. Mézard,et al.  Two Solutions to Diluted p-Spin Models and XORSAT Problems , 2003 .

[6]  Martin J. Wainwright,et al.  Low density codes achieve the rate-distortion bound , 2006, Data Compression Conference (DCC'06).

[7]  R. Zecchina,et al.  Message-Passing Algorithms for Non-Linear Nodes and Data Compression , 2005, Complexus.

[8]  Nadia Creignou,et al.  Approximating The Satisfiability Threshold For Random K-Xor-Formulas , 2003, Comb. Probab. Comput..

[9]  Riccardo Zecchina,et al.  Alternative solutions to diluted p-spin models and XORSAT problems , 2002, ArXiv.

[10]  K. Ramchandran,et al.  Turbo coded trellis-based constructions for data embedding: channel coding with side information , 2001, Conference Record of Thirty-Fifth Asilomar Conference on Signals, Systems and Computers (Cat.No.01CH37256).

[11]  T. Murayama.  Thouless-Anderson-Palmer approach for lossy compression. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[12]  Martin J. Wainwright,et al.  Lossy source encoding via message-passing and decimation over generalized codewords of LDGM codes , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[13]  Zixiang Xiong,et al.  On Multiterminal Source Code Design , 2005, IEEE Transactions on Information Theory.

[14]  Zixiang Xiong,et al.  On Multiterminal Source Code Design , 2008, IEEE Trans. Inf. Theory.

[15]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[16]  Kannan Ramchandran,et al.  Turbo and trellis-based constructions for source coding with side information , 2003, Data Compression Conference, 2003. Proceedings. DCC 2003.

[17]  Stephan ten Brink,et al.  A close-to-capacity dirty paper coding scheme , 2004, IEEE Transactions on Information Theory.

[18]  Michael W. Marcellin,et al.  Trellis coded quantization of memoryless and Gauss-Markov sources , 1990, IEEE Trans. Commun..

[19]  Stephan ten Brink,et al.  A close-to-capacity dirty paper coding scheme , 2004, ISIT.

[20]  R. Monasson,et al.  Rigorous decimation-based construction of ground pure states for spin-glass models on random lattices. , 2002, Physical review letters.