Joint Empirical Coordination of Source and Channel

In a decentralized and self-configuring network, the communication devices are considered as autonomous decision-makers that sense their environment and that implement optimal transmission schemes. It is essential that these autonomous devices cooperate and coordinate their actions, to ensure the reliability of the transmissions and the stability of the network. We study a point-to-point scenario in which the encoder and the decoder implement decentralized policies that are coordinated. The coordination is measured in terms of empirical frequency of symbols of source and channel. The encoder and the decoder perform a coding scheme such that the empirical distribution of the symbols is close to a target joint probability distribution. We characterize the set of achievable target probability distributions for a point-to-point source-channel model, in which the encoder is non-causal and the decoder is strictly causal i.e., it returns an action based on the observation of the past channel outputs. The objectives of the encoder and of the decoder, are captured by some utility function, evaluated with respect to the set of achievable target probability distributions. In this article, we investigate the maximization problem of a utility function that is common to both encoder and decoder. We show that the compression and the transmission of information are particular cases of the empirical coordination.

[1]  Maël Le Treust,et al.  Information design for strategic coordination of autonomous devices with non-aligned utilities , 2016, 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[2]  Sergio Verdú,et al.  Approximation theory of output statistics , 1993, IEEE Trans. Inf. Theory.

[3]  Robert J. Aumann,et al.  Repeated Games with Incomplete Information , 1995 .

[4]  Haim H. Permuter,et al.  Coordination Capacity , 2009, IEEE Transactions on Information Theory.

[5]  Samson Lasaulce,et al.  An achievable rate region for the broadcast wiretap channel with asymmetric side information , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[6]  R. M. Dudley,et al.  Real Analysis and Probability , 1989 .

[7]  Matthieu R. Bloch,et al.  Strong Coordination of Signals and Actions Over Noisy Channels With Two-Sided State Information , 2018, IEEE Transactions on Information Theory.

[8]  Mung Chiang,et al.  Channel capacity and state estimation for state-dependent Gaussian channels , 2005, IEEE Transactions on Information Theory.

[9]  Shlomo Shamai,et al.  The empirical distribution of good codes , 1997, IEEE Trans. Inf. Theory.

[10]  Alon Orlitsky,et al.  Coding for computing , 2001, IEEE Trans. Inf. Theory.

[11]  Samson Lasaulce,et al.  Coordination in state-dependent distributed networks: The two-agent case , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[12]  Samson Lasaulce,et al.  Coordinating partially-informed agents over state-dependent networks , 2015, 2015 IEEE Information Theory Workshop (ITW).

[13]  Peter W. Shor,et al.  Entanglement-assisted capacity of a quantum channel and the reverse Shannon theorem , 2001, IEEE Trans. Inf. Theory.

[14]  Haim H. Permuter,et al.  The Ahlswede-Körner coordination problem with one-sided encoder cooperation , 2014, 2014 IEEE International Symposium on Information Theory.

[15]  Samson Lasaulce,et al.  A Repeated Game Formulation of Energy-Efficient Decentralized Power Control , 2010, IEEE Transactions on Wireless Communications.

[16]  Mérouane Debbah,et al.  Power allocation games for mimo multiple access channels with coordination , 2009, IEEE Transactions on Wireless Communications.

[17]  Urbashi Mitra,et al.  Capacity-distortion trade-off in channels with state , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[18]  Jörg Kliewer,et al.  Strong coordination over a line when actions are Markovian , 2016, 2016 Annual Conference on Information Science and Systems (CISS).

[19]  Sung Hoon Lim,et al.  Lossy communication of correlated sources over multiple access channels , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[20]  Zhu Han,et al.  The Nash equilibrium region of the linear deterministic interference channel with feedback , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[21]  Tsachy Weissman,et al.  The empirical distribution of rate-constrained source codes , 2004, IEEE Transactions on Information Theory.

[22]  Paul W. Cuff,et al.  Hybrid codes needed for coordination over the point-to-point channel , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[23]  Mikael Skoglund,et al.  Polar Codes for Coordination in Cascade Networks , 2012 .

[24]  Maxim Raginsky,et al.  Empirical Processes, Typical Sequences, and Coordinated Actions in Standard Borel Spaces , 2010, IEEE Transactions on Information Theory.

[25]  Mael Le Treust,et al.  Empirical coordination with two-sided state information and correlated source and state , 2015, ISIT.

[26]  Jörg Kliewer,et al.  Strong coordination over a three-terminal relay network , 2014, 2014 IEEE Information Theory Workshop (ITW 2014).

[27]  Aaron D. Wyner,et al.  The rate-distortion function for source coding with side information at the decoder , 1976, IEEE Trans. Inf. Theory.

[28]  Haim H. Permuter,et al.  Capacity of Coordinated Actions , 2007, 2007 IEEE International Symposium on Information Theory.

[29]  Sergio Verdú,et al.  Total variation distance and the distribution of relative information , 2014, 2014 Information Theory and Applications Workshop (ITA).

[30]  Jörg Kliewer,et al.  Polar coding for empirical and strong coordination via distribution approximation , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[31]  Sylvain Sorin,et al.  Repeated Games by Jean-François Mertens , 2015 .

[32]  Maël Le Treust.  Théorie de l’information, jeux répétés avec observation imparfaite et réseaux de communication décentralisés , 2011 .

[33]  Serap A. Savari,et al.  Quantum data compression with commuting density operators , 2002, Proceedings IEEE International Symposium on Information Theory,.

[34]  Vivek S. Borkar,et al.  Common randomness and distributed control: A counterexample , 2007, Systems & control letters (Print).

[35]  Venkat Anantharam,et al.  Generating dependent random variables over networks , 2011, 2011 IEEE Information Theory Workshop.

[36]  Penélope Hernández,et al.  Optimal use of communication resources , 2006 .

[37]  Haim H. Permuter,et al.  Successive Refinement With Decoder Cooperation and Its Channel Coding Duals , 2012, IEEE Transactions on Information Theory.

[38]  Urbashi Mitra,et al.  Causal State Communication , 2012, IEEE Transactions on Information Theory.

[39]  Abbas El Gamal,et al.  Network Information Theory , 2021, 2021 IEEE 3rd International Conference on Advanced Trends in Information Theory (ATIT).

[40]  Nicolas Vieille,et al.  How to play with a biased coin? , 2002, Games Econ. Behav..

[41]  Mung Chiang,et al.  Duality between channel capacity and rate distortion with two-sided state information , 2002, IEEE Trans. Inf. Theory.

[42]  Jörg Kliewer,et al.  Strong coordination with polar codes , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[43]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[44]  Serap A. Savari,et al.  Communicating Probability Distributions , 2007, IEEE Transactions on Information Theory.

[45]  Matthieu R. Bloch,et al.  Polar coding for empirical coordination of signals and actions over noisy channels , 2016, 2016 IEEE Information Theory Workshop (ITW).

[46]  Rudolf Ahlswede,et al.  Source coding with side information and a converse for degraded broadcast channels , 1975, IEEE Trans. Inf. Theory.

[47]  Paul W. Cuff,et al.  Distributed Channel Synthesis , 2012, IEEE Transactions on Information Theory.

[48]  Urbashi Mitra,et al.  On non-causal side information at the encoder , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[49]  Paul W. Cuff,et al.  Secure cascade channel synthesis , 2013, ISIT.

[50]  Matthieu R. Bloch,et al.  Empirical coordination, state masking and state amplification: Core of the decoder's knowledge , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[51]  Matthieu R. Bloch,et al.  Coordination in Distributed Networks via Coded Actions With Application to Power Control , 2015, IEEE Transactions on Information Theory.

[52]  Ali Bereyhi,et al.  Empirical coordination in a triangular multiterminal network , 2013, 2013 IEEE International Symposium on Information Theory.

[53]  Lei Zhao,et al.  Coordination using implicit communication , 2011, 2011 IEEE Information Theory Workshop.

[54]  Samson Lasaulce,et al.  Coded power control: Performance analysis , 2013, 2013 IEEE International Symposium on Information Theory.

[55]  Rida Laraki,et al.  Informationally optimal correlation , 2008, Math. Program..

[56]  Shlomo Shamai,et al.  On joint source-channel coding for the Wyner-Ziv source and the Gel'fand-Pinsker channel , 2003, IEEE Trans. Inf. Theory.

[57]  Sung Hoon Lim,et al.  Joint source-channel coding via hybrid coding , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[58]  Paul W. Cuff,et al.  Communication requirements for generating correlated random variables , 2008, 2008 IEEE International Symposium on Information Theory.

[59]  Mael Le Treust.  Empirical coordination with channel feedback and strictly causal or causal encoding , 2015, ISIT.

[60]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[61]  Maël Le Treust.  Correlation between channel state and information source with empirical coordination constraint , 2014, 2014 IEEE Information Theory Workshop (ITW 2014).

[62]  Tristan Tomala,et al.  Empirical Distributions of Beliefs Under Imperfect Observation , 2006, Math. Oper. Res..

[63]  Urbashi Mitra,et al.  Causal state amplification , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[64]  Samson Lasaulce,et al.  Transforming Monitoring Structures with Resilient Encoders—Application to Repeated Games , 2012, Dyn. Games Appl..

[65]  Jörg Kliewer,et al.  Strong coordination over multi-hop line networks , 2015, 2015 IEEE Information Theory Workshop - Fall (ITW).

[66]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[67]  Sidharth Jaggi,et al.  Zero error coordination , 2015, 2015 IEEE Information Theory Workshop - Fall (ITW).

[68]  Maxim Raginsky,et al.  Empirical processes and typical sequences , 2010, 2010 IEEE International Symposium on Information Theory.

[69]  David Tse,et al.  Shannon Meets Nash on the Interference Channel , 2010, IEEE Transactions on Information Theory.

[70]  Jörg Kliewer,et al.  Strong coordination over a line network , 2013, 2013 IEEE International Symposium on Information Theory.

[71]  Aaron D. Wyner,et al.  The common information of two dependent random variables , 1975, IEEE Trans. Inf. Theory.

[72]  Tristan Tomala,et al.  Secret Correlation in Repeated Games with Imperfect Monitoring , 2007, Math. Oper. Res..

[73]  Amin Gohari,et al.  Coordination via a relay , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.