One step RSB scheme for the rate distortion function

We apply statistical mechanics to an inverse problem of linear mapping to investigate the physics of the irreversible compression. We use the replica symmetry breaking (RSB) technique with a toy model to demonstrate the Shannon result. The rate distortion function, which is widely known as the theoretical limit of the compression with a fidelity criterion, is derived using the Parisi one step RSB scheme. The bound cannot be achieved in the sparsely-connected systems, where suboptimal solutions dominate the capacity.

[1]  Y. Kabashima,et al.  Statistical mechanics of lossy data compression using a nonmonotonic perceptron. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[2]  H. Yamamoto,et al.  A coding theorem for lossy data compression by LDPC codes , 2002, Proceedings IEEE International Symposium on Information Theory,.

[3]  Tatsuto Murayama.  Statistical mechanics of the data compression theorem , 2002 .

[4]  P. Kurasov,et al.  On the inverse scattering problem on branching graphs , 2002 .

[5]  D. Stein.  Introduction to the Replica Theory of Disordered Statistical Systems , 2002 .

[6]  H. Nishimori.  Statistical Physics of Spin Glasses and Information Processing , 2001 .

[7]  Riccardo Zecchina,et al.  Exact solutions for diluted spin glasses and optimization problems , 2001, Physical review letters.

[8]  V. Dotsenko.  Introduction to the Replica Theory of Disordered Statistical Systems: Two-dimensional Ising model with disorder , 2005 .

[9]  L. Mankiewicz,et al.  Power corrections to the process $\gamma\gamma^*\rightarrow \pi\pi$ in the light-cone sum rules approach , 2000, hep-ph/0008168.

[10]  Saad,et al.  Typical performance of gallager-type error-correcting codes , 1999, Physical review letters.

[11]  Andrea Montanari,et al.  The statistical mechanics of turbo codes , 1999 .

[12]  D. Saad,et al.  Statistical mechanics of error-correcting codes , 1999 .

[13]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[14]  Nicolas Sourlas,et al.  Spin-glass models as error-correcting codes , 1989, Nature.

[15]  D. Sherrington,et al.  Graph bipartitioning and spin glasses on a random network of fixed finite valence , 1987 .

[16]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[17]  B. Derrida.  Random-energy model: An exactly solvable model of disordered systems , 1981 .

[18]  Toby Berger,et al.  Rate distortion theory : a mathematical basis for data compression , 1971 .