A Scaling Theory for Distributed Sparse Matrix Coding in the CEO Problem

We consider the problem of distributed sensing in a noisy environment; the CEO problem. Here individual sensor readings are encoded and transmitted independently by multiple sensors with a limited combined data rate. We present a scaling analysis of a semi-practical system using the sparse matrix codes, and show that the analysis is consistent with a general argument based on an existence of the rate distortion function. This approach well describes the tradeoff between reducing errors due to environmental noise and increasing errors due to lossy coding as the number of sensors increases, showing threshold behavior for optimal number of sensors

[1]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[2]  Tatsuto Murayama On the asymptotic properties of sparse matrix codes in the CEO problem , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[3]  Viktor Dotsenko,et al.  Introduction to the Replica Theory of Disordered Statistical Systems: Introduction , 2000 .

[4]  T. Murayama Thouless-Anderson-Palmer approach for lossy compression. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[5]  Emin Martinian,et al.  Iterative Quantization Using Codes On Graphs , 2004, ArXiv.

[6]  Zhen Zhang,et al.  On the CEO problem , 1994, Proceedings of 1994 IEEE International Symposium on Information Theory.

[7]  Martin J. Wainwright,et al.  Lossy source encoding via message-passing and decimation over generalized codewords of LDGM codes , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[8]  Masato Okada,et al.  Rate Distortion Function in the Spin Glass State: A Toy Model , 2002, NIPS.

[9]  Nicolas Sourlas,et al.  Spin-glass models as error-correcting codes , 1989, Nature.

[10]  L. Goddard Information Theory , 1962, Nature.

[11]  Kai Lai Chung,et al.  A Course in Probability Theory , 1949 .

[12]  Toby Berger,et al.  The CEO problem [multiterminal source coding] , 1996, IEEE Trans. Inf. Theory.