Rate Distortion Function in the Spin Glass State: A Toy Model

We applied statistical mechanics to an inverse problem of linear mapping to investigate the physics of optimal lossy compressions. We used the replica symmetry breaking technique with a toy model to demonstrate Shannon's result. The rate distortion function, which is widely known as the theoretical limit of the compression with a fidelity criterion, is derived. Numerical study shows that sparse constructions of the model provide suboptimal compressions.

[1]  Aaron D. Wyner,et al.  Coding Theorems for a Discrete Source With a Fidelity CriterionInstitute of Radio Engineers, International Convention Record, vol. 7, 1959. , 1993 .

[2]  Bernard Derrida,et al.  The random energy model , 1980 .

[3]  Y. Kabashima,et al.  Statistical mechanics of lossy data compression using a nonmonotonic perceptron. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[4]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[5]  D. Sherrington,et al.  Graph bipartitioning and spin glasses on a random network of fixed finite valence , 1987 .

[6]  D. Saad,et al.  Statistical mechanics of error-correcting codes , 1999 .

[7]  H. Yamamoto,et al.  A coding theorem for lossy data compression by LDPC codes , 2002, Proceedings IEEE International Symposium on Information Theory,.

[8]  Toby Berger,et al.  Rate distortion theory : a mathematical basis for data compression , 1971 .

[9]  Andrea Montanari,et al.  The statistical mechanics of turbo codes , 1999 .

[10]  B. Derrida.  Random-energy model: An exactly solvable model of disordered systems , 1981 .

[11]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[12]  H. Nishimori.  Statistical Physics of Spin Glasses and Information Processing , 2001 .

[13]  Saad,et al.  Typical performance of gallager-type error-correcting codes , 2000, Physical review letters.

[14]  Tatsuto Murayama,et al.  Statistical mechanics of the data compression theorem , 2002 .

[15]  Viktor Dotsenko,et al.  Introduction to the Replica Theory of Disordered Statistical Systems: Introduction , 2000 .

[16]  Nicolas Sourlas,et al.  Spin-glass models as error-correcting codes , 1989, Nature.