The theoretical capacity of the Parity Source Coder

The Parity Source Coder is a protocol for data compression which is based on a set of parity checks organized in a sparse random network. We consider here the case of memoryless unbiased binary sources. We show that the theoretical capacity saturates the Shannon limit at large K. We also find that the first corrections to the leading behaviour are exponentially small, with the result that the behaviour at finite K is very close to the optimal one.

[1]  T. Murayama.  Thouless-Anderson-Palmer approach for lossy compression. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[2]  Y. Kabashima,et al.  Statistical mechanics of lossy data compression using a nonmonotonic perceptron. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  Yoshiyuki Kabashima,et al.  Statistical Mechanical Approach to Error Exponents of Lossy Data Compression , 2003 .

[4]  M. Mézard,et al.  Random K-satisfiability problem: from an analytic solution to an efficient algorithm. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[5]  D. Saad,et al.  Statistical mechanics of error-correcting codes , 1999 .

[6]  R. Monasson,et al.  Rigorous decimation-based construction of ground pure states for spin-glass models on random lattices. , 2002, Physical review letters.

[7]  M. Mézard,et al.  Two Solutions to Diluted p-Spin Models and XORSAT Problems , 2003 .

[8]  Riccardo Zecchina,et al.  Survey propagation as local equilibrium equations , 2003, ArXiv.

[9]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[10]  Masato Okada,et al.  One step RSB scheme for the rate distortion function , 2002, cond-mat/0207637.

[11]  D. Vernon.  Inform , 1995, Encyclopedia of the UN Sustainable Development Goals.

[12]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[13]  Oliver Johnson,et al.  International Symposium on Information Theory , 2007 .

[14]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[15]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[16]  B. Derrida.  Random-energy model: An exactly solvable model of disordered systems , 1981 .

[17]  M. Mézard,et al.  The simplest spin glass , 1984 .

[18]  M. Chalmers.  V. Conclusions , 1986 .

[19]  L. Goddard.  Information Theory , 1962, Nature.

[20]  Riccardo Zecchina,et al.  Lossy data compression with random gates. , 2005, Physical review letters.

[21]  Andrea Montanari,et al.  Instability of one-step replica-symmetry-broken phase in satisfiability problems , 2003, ArXiv.