Empirical Coordination for Joint Source-Channel Coding

We investigate the problem of simultaneous transmission and coordination for a point-to-point communication model. The encoder and decoder choose their symbols/actions (i.e. channel input and decoder output) in order to transmit information and to coordinate themselves. This paradigm has promising repercussions for future decentralized networks. Exploiting the coordination capabilities, devices can implement decentralized policies that are coordinated. Exploiting transmission capabilities, devices choose their actions (power control, the channel allocation) in order to encode additional embedded information (channel state information, future allocations). The feasible trade-offs between transmissions and coordinations are characterized using joint probability distributions over the source and channel symbols. Encoder and decoder implement a coding scheme such that the empirical frequency of symbols are close to target joint probability distribution. We characterize the set of achievable probability distribution for a point-to-point source-channel model with strictly-causal and causal decoding. Compared to classical coding results, the coordination requirement induces a stronger information constraint. We determine optimal trade-offs between transmission and coordination by maximizing a utility function over the set of achievable probability distribution.

[1]  Sergio Verdú,et al.  Approximation theory of output statistics , 1993, IEEE Trans. Inf. Theory.

[2]  Robert J. Aumann,et al.  Repeated Games with Incomplete Information , 1995 .

[3]  Zhu Han,et al.  The Nash equilibrium region of the linear deterministic interference channel with feedback , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[4]  Paul W. Cuff,et al.  Hybrid codes needed for coordination over the point-to-point channel , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[5]  Urbashi Mitra,et al.  On non-causal side information at the encoder , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[6]  R. M. Dudley,et al.  Real Analysis and Probability , 1989 .

[7]  Maël Le Treust.  Correlation between channel state and information source with empirical coordination constraint , 2014, 2014 IEEE Information Theory Workshop (ITW 2014).

[8]  Maxim Raginsky,et al.  Empirical Processes, Typical Sequences, and Coordinated Actions in Standard Borel Spaces , 2010, IEEE Transactions on Information Theory.

[9]  Paul W. Cuff,et al.  Distributed Channel Synthesis , 2012, IEEE Transactions on Information Theory.

[10]  Tristan Tomala,et al.  Empirical Distributions of Beliefs Under Imperfect Observation , 2006, Math. Oper. Res..

[11]  Jörg Kliewer,et al.  Polar coding for empirical and strong coordination via distribution approximation , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[12]  L. Goddard.  Information Theory , 1962, Nature.

[13]  Venkat Anantharam,et al.  Generating dependent random variables over networks , 2011, 2011 IEEE Information Theory Workshop.

[14]  Samson Lasaulce,et al.  Transforming Monitoring Structures with Resilient Encoders—Application to Repeated Games , 2012, Dyn. Games Appl..

[15]  Haim H. Permuter,et al.  Coordination Capacity , 2009, IEEE Transactions on Information Theory.

[16]  Samson Lasaulce,et al.  An achievable rate region for the broadcast wiretap channel with asymmetric side information , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[17]  Urbashi Mitra,et al.  Causal State Communication , 2012, IEEE Transactions on Information Theory.

[18]  Vivek S. Borkar,et al.  Common randomness and distributed control: A counterexample , 2007, Systems & control letters (Print).

[19]  Ali Bereyhi,et al.  Empirical coordination in a triangular multiterminal network , 2013, 2013 IEEE International Symposium on Information Theory.

[20]  Lei Zhao,et al.  Coordination using implicit communication , 2011, 2011 IEEE Information Theory Workshop.

[21]  Aaron D. Wyner,et al.  The rate-distortion function for source coding with side information at the decoder , 1976, IEEE Trans. Inf. Theory.

[22]  Samson Lasaulce,et al.  Coded power control: Performance analysis , 2013, 2013 IEEE International Symposium on Information Theory.

[23]  Mikael Skoglund,et al.  Polar Codes for Coordination in Cascade Networks , 2012 .

[24]  Penélope Hernández,et al.  Optimal use of communication resources , 2006 .

[25]  Haim H. Permuter,et al.  Successive Refinement With Decoder Cooperation and Its Channel Coding Duals , 2012, IEEE Transactions on Information Theory.

[26]  Sidharth Jaggi,et al.  Zero error coordination , 2015, 2015 IEEE Information Theory Workshop - Fall (ITW).

[27]  Rida Laraki,et al.  Informationally optimal correlation , 2008, Math. Program..

[28]  Maxim Raginsky,et al.  Empirical processes and typical sequences , 2010, 2010 IEEE International Symposium on Information Theory.

[29]  David Tse,et al.  Shannon Meets Nash on the Interference Channel , 2010, IEEE Transactions on Information Theory.

[30]  Shlomo Shamai,et al.  On joint source-channel coding for the Wyner-Ziv source and the Gel'fand-Pinsker channel , 2003, IEEE Trans. Inf. Theory.

[31]  Mung Chiang,et al.  Duality between channel capacity and rate distortion with two-sided state information , 2002, IEEE Trans. Inf. Theory.

[32]  Sung Hoon Lim,et al.  Joint source-channel coding via hybrid coding , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[33]  Jörg Kliewer,et al.  Strong coordination with polar codes , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[34]  Jörg Kliewer,et al.  Strong coordination over a three-terminal relay network , 2014, 2014 IEEE Information Theory Workshop (ITW 2014).

[35]  Rudolf Ahlswede,et al.  Source coding with side information and a converse for degraded broadcast channels , 1975, IEEE Trans. Inf. Theory.

[36]  Alon Orlitsky,et al.  Coding for computing , 2001, IEEE Trans. Inf. Theory.

[37]  Samson Lasaulce,et al.  Coordination in state-dependent distributed networks: The two-agent case , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[38]  Urbashi Mitra,et al.  Capacity-distortion trade-off in channels with state , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[39]  Sung Hoon Lim,et al.  Lossy communication of correlated sources over multiple access channels , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[40]  Paul W. Cuff,et al.  Secure cascade channel synthesis , 2013, ISIT.

[41]  Peter W. Shor,et al.  Entanglement-assisted capacity of a quantum channel and the reverse Shannon theorem , 2001, IEEE Trans. Inf. Theory.

[42]  Haim H. Permuter,et al.  The Ahlswede-Körner coordination problem with one-sided encoder cooperation , 2014, 2014 IEEE International Symposium on Information Theory.

[43]  Samson Lasaulce,et al.  A Repeated Game Formulation of Energy-Efficient Decentralized Power Control , 2010, IEEE Transactions on Wireless Communications.

[44]  Mérouane Debbah,et al.  Power allocation games for mimo multiple access channels with coordination , 2009, IEEE Transactions on Wireless Communications.

[45]  Jörg Kliewer,et al.  Strong coordination over multi-hop line networks , 2015, 2015 IEEE Information Theory Workshop - Fall (ITW).

[46]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[47]  Jörg Kliewer,et al.  Strong coordination over a line network , 2013, 2013 IEEE International Symposium on Information Theory.

[48]  Aaron D. Wyner,et al.  The common information of two dependent random variables , 1975, IEEE Trans. Inf. Theory.

[49]  Tristan Tomala,et al.  Secret Correlation in Repeated Games with Imperfect Monitoring , 2007, Math. Oper. Res..

[50]  Amin Gohari,et al.  Coordination via a relay , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[51]  Abbas El Gamal,et al.  Network Information Theory , 2021, 2021 IEEE 3rd International Conference on Advanced Trends in Information Theory (ATIT).

[52]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[53]  Haim H. Permuter,et al.  Capacity of Coordinated Actions , 2007, 2007 IEEE International Symposium on Information Theory.

[54]  Sergio Verdú,et al.  Total variation distance and the distribution of relative information , 2014, 2014 Information Theory and Applications Workshop (ITA).

[55]  Serap A. Savari,et al.  Quantum data compression with commuting density operators , 2002, Proceedings IEEE International Symposium on Information Theory,.

[56]  Shlomo Shamai,et al.  The empirical distribution of good codes , 1997, IEEE Trans. Inf. Theory.

[57]  Mung Chiang,et al.  Channel capacity and state estimation for state-dependent Gaussian channels , 2005, IEEE Transactions on Information Theory.

[58]  Paul W. Cuff,et al.  Communication requirements for generating correlated random variables , 2008, 2008 IEEE International Symposium on Information Theory.

[59]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[60]  Serap A. Savari,et al.  Communicating Probability Distributions , 2007, IEEE Transactions on Information Theory.

[61]  Nicolas Vieille,et al.  How to play with a biased coin? , 2002, Games Econ. Behav..