On the Structure of Real-Time Encoders and Decoders in a Multi-Terminal Communication System

A real-time communication system with two encoders communicating with a single receiver over separate noisy channels is considered. The two encoders make distinct partial observations of a Markov source. Each encoder must encode its observations into a sequence of discrete symbols. The symbols are transmitted over noisy channels to a finite memory receiver that attempts to reconstruct some function of the state of the Markov source. Encoding and decoding must be done in real-time, that is, the distortion measure does not tolerate delays. Under the assumption that the encoders' observations are conditionally independent Markov chains given an unobserved time-invariant random variable, results on the structure of optimal real-time encoders and the receiver are obtained. It is shown that there exist finite-dimensional sufficient statistics for the encoders. The problem with noiseless channels and perfect memory at the receiver is then considered. A new methodology to find the structure of optimal real-time encoders is employed. A sufficient statistic with a time-invariant domain is found for this problem. This methodology exploits the presence of common information between the encoders and the receiver when communication is over noiseless channels.

[1]  Te Sun Han,et al.  A dichotomy of functions F(X, Y) of correlated sources (X, Y) , 1987, IEEE Trans. Inf. Theory.

[2]  S. Sandeep Pradhan,et al.  Lattices for Distributed Source Coding: Jointly Gaussian Sources and Reconstruction of a Linear Function , 2007, AAECC.

[3]  D. Slepian,et al.  A coding theorem for multiple access channels with correlated sources , 1973 .

[4]  V. Borkar,et al.  Optimal sequential vector quantization of Markov sources , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[5]  Toby Berger,et al.  An upper bound on the sum-rate distortion function and its corresponding rate allocation schemes for the CEO problem , 2004, IEEE Journal on Selected Areas in Communications.

[6]  Hans S. Witsenhausen,et al.  A standard form for sequential stochastic control , 1973, Mathematical systems theory.

[7]  Yasutada Oohama,et al.  The Rate-Distortion Function for the Quadratic Gaussian CEO Problem , 1998, IEEE Trans. Inf. Theory.

[8]  Michelle Effros,et al.  Optimal code design for lossless and near lossless source coding in multiple access networks , 2001, Proceedings DCC 2001. Data Compression Conference.

[9]  N. THOMAS GAARDER,et al.  On optimal finite-state digital transmission systems , 1982, IEEE Trans. Inf. Theory.

[10]  T. Berger,et al.  The quadratic Gaussian CEO problem , 1995, Proceedings of 1995 IEEE International Symposium on Information Theory.

[11]  Demosthenis Teneketzis,et al.  Sequential decomposition of sequential dynamic teams: applications to real-time communication and networked control systems , 2008 .

[12]  Ashutosh Nayyar,et al.  On globally optimal real-time encoding and decoding strategies in multi-terminal communication systems , 2008, 2008 47th IEEE Conference on Decision and Control.

[13]  Pravin Varaiya,et al.  Stochastic Systems: Estimation, Identification, and Adaptive Control , 1986 .

[14]  R. Aumann Agreeing to disagree. , 1976, Nature cell biology.

[15]  R. A. McDonald,et al.  Noiseless Coding of Correlated Information Sources , 1973 .

[16]  A. Kh. Al Jabri,et al.  Zero-Error Codes for Correlated Information Sources , 1997, IMACC.

[17]  H. Witsenhausen On the structure of real-time source coders , 1979, The Bell System Technical Journal.

[18]  Y. Oohama Gaussian multiterminal source coding , 1995, Proceedings of 1995 IEEE International Symposium on Information Theory.

[19]  H. Witsenhausen A Counterexample in Stochastic Optimum Control , 1968 .

[20]  Robert M. Gray,et al.  Encoding of correlated observations , 1987, IEEE Trans. Inf. Theory.

[21]  Masoud Salehi,et al.  Multiple access channels with arbitrarily correlated sources , 1980, IEEE Trans. Inf. Theory.

[22]  Zhen Zhang,et al.  On the CEO problem , 1994, Proceedings of 1994 IEEE International Symposium on Information Theory.

[23]  Kannan Ramchandran,et al.  Distributed source coding using syndromes (DISCUSS): design and construction , 1999 .

[24]  Demosthenis Teneketzis,et al.  On the Structure of Optimal Real-Time Encoders and Decoders in Noisy Communication , 2006, IEEE Transactions on Information Theory.

[25]  Yu-Chi Ho,et al.  Team decision theory and information structures , 1980 .

[26]  Tamer Basar,et al.  Causal coding of markov sources with continuous alphabets , 2008 .

[27]  Toby Berger,et al.  Multiterminal Source Coding with High Resolution , 1999, IEEE Trans. Inf. Theory.

[28]  D. Teneketzis,et al.  Identifying tractable decentralized control problems on the basis of information structure , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[29]  K. Ramchandran,et al.  Distributed source coding using syndromes (DISCUS): design and construction , 1999, Proceedings DCC'99 Data Compression Conference (Cat. No. PR00096).

[30]  Rudolf Ahlswede,et al.  On source coding with side information via a multiple-access channel and related problems in multi-user information theory , 1983, IEEE Trans. Inf. Theory.

[31]  R. Gallager Information Theory and Reliable Communication , 1968 .

[32]  Demosthenis Teneketzis,et al.  On the design of globally optimal communication strategies for real-time noisy communication systems with noisy feedback , 2008, IEEE Journal on Selected Areas in Communications.

[33]  Stark C. Draper,et al.  Successively structured CEO problems , 2002, Proceedings IEEE International Symposium on Information Theory,.

[34]  János Körner,et al.  How to encode the modulo-two sum of binary sources (Corresp.) , 1979, IEEE Trans. Inf. Theory.

[35]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[36]  Jean C. Walrand,et al.  Optimal causal coding - decoding problems , 1983, IEEE Trans. Inf. Theory.

[37]  Demosthenis Teneketzis,et al.  On Globally Optimal Encoding, Decoding, and Memory Update for Noisy Real-Time Communication Systems , 2008 .