Optimality and Approximate Optimality of Source-Channel Separation in Networks

We consider the source-channel separation architecture for lossy source coding in communication networks. It is shown that the separation approach is optimal in two general scenarios and is approximately optimal in a third scenario. The two scenarios for which separation is optimal complement each other: the first is when the memoryless sources at source nodes are arbitrarily correlated, each of which is to be reconstructed at possibly multiple destinations within certain distortions, but the channels in this network are synchronized, orthogonal, and memoryless point-to-point channels; the second is when the memoryless sources are mutually independent, each of which is to be reconstructed only at one destination within a certain distortion, but the channels are general, including multi-user channels, such as multiple access, broadcast, interference, and relay channels, possibly with feedback. The third scenario, for which we demonstrate approximate optimality of source-channel separation, generalizes the second scenario by allowing each source to be reconstructed at multiple destinations with different distortions. For this case, the loss from optimality using the separation approach can be upper-bounded when a difference distortion measure is taken, and in the special case of quadratic distortion measure, this leads to universal constant bounds.

[1]  Abbas El Gamal,et al.  Achievable rates for multiple descriptions , 1982, IEEE Trans. Inf. Theory.

[2]  Abbas El Gamal,et al.  Network Information Theory , 2021, 2021 IEEE 3rd International Conference on Advanced Trends in Information Theory (ATIT).

[3]  Rudolf Ahlswede,et al.  Network information flow , 2000, IEEE Trans. Inf. Theory.

[4]  Gerhard Kramer,et al.  The multicast capacity of deterministic relay networks with no interference , 2006, IEEE Transactions on Information Theory.

[5]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[6]  Shlomo Shamai,et al.  On source-channel separation in networks , 2010, 2010 International Conference on Signal Processing and Communications (SPCOM).

[7]  Masoud Salehi,et al.  Multiple access channels with arbitrarily correlated sources , 1980, IEEE Trans. Inf. Theory.

[8]  João Barros,et al.  Network information flow with correlated sources , 2005, IEEE Transactions on Information Theory.

[9]  Bixio Rimoldi,et al.  Successive refinement of information: characterization of the achievable rates , 1994, IEEE Trans. Inf. Theory.

[10]  Stark C. Draper,et al.  Side information aware coding strategies for sensor networks , 2004, IEEE Journal on Selected Areas in Communications.

[11]  Amiram H. Kaspi,et al.  Two-way source coding with a fidelity criterion , 1985, IEEE Trans. Inf. Theory.

[12]  Sriram Vishwanath,et al.  Hybrid coding for Gaussian broadcast channels with Gaussian sources , 2009, 2009 IEEE International Symposium on Information Theory.

[13]  Shirin Jalali,et al.  Separation of Source-Network Coding and Channel Coding in Wireline Networks , 2011, IEEE Transactions on Information Theory.

[14]  Anant Sahai,et al.  Coding into a source: a direct inverse Rate-Distortion theorem , 2006, ArXiv.

[15]  Aaron D. Wyner,et al.  The rate-distortion function for source coding with side information at the decoder , 1976, IEEE Trans. Inf. Theory.

[16]  Toby Berger,et al.  Rate distortion theory : a mathematical basis for data compression , 1971 .

[17]  Alon Orlitsky,et al.  Average-case interactive communication , 1992, IEEE Trans. Inf. Theory.

[18]  Te Sun Han,et al.  Slepian-Wolf-Cover Theorem for Networks of Channels , 1980, Inf. Control..

[19]  Shirin Jalali,et al.  On the separation of lossy source-network coding and channel coding in wireline networks , 2010, 2010 IEEE International Symposium on Information Theory.

[20]  Shlomo Shamai,et al.  Optimality and Approximate Optimality of Source-Channel Separation in Networks , 2014, IEEE Trans. Inf. Theory.

[21]  János Körner,et al.  General broadcast channels with degraded message sets , 1977, IEEE Trans. Inf. Theory.

[22]  Haim H. Permuter,et al.  Cascade and triangular source coding with side information at the first two nodes , 2010, ISIT.

[23]  Wei Liu,et al.  Communicating Correlated Gaussian Sources over Gaussian Z Interference Channels , 2009, GLOBECOM 2009 - 2009 IEEE Global Telecommunications Conference.

[24]  Hirosuke Yamamoto,et al.  Source coding theory for cascade and branching communication systems , 1981, IEEE Trans. Inf. Theory.

[25]  Suhas N. Diggavi,et al.  Approximate characterizations for the Gaussian broadcasting distortion region , 2009, 2009 IEEE International Symposium on Information Theory.

[26]  Muriel Médard,et al.  On a theory of network equivalence , 2009, 2009 IEEE Information Theory Workshop on Networking and Information Theory.

[27]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[28]  David Tse,et al.  Approximating the rate-distortion region of the distributed source coding for three jointly Gaussian tree-structured sources , 2009, 2009 IEEE International Symposium on Information Theory.

[29]  Ertem Tuncel,et al.  Slepian-Wolf coding over broadcast channels , 2006, IEEE Transactions on Information Theory.

[30]  Ram Zamir,et al.  The rate loss in the Wyner-Ziv problem , 1996, IEEE Trans. Inf. Theory.

[31]  Lossy source coding for a cascade communication system with side-informations , 2006 .

[32]  R. Gallager Information Theory and Reliable Communication , 1968 .

[33]  Toby Berger,et al.  Rate-distortion for correlated sources with partially separated encoders , 1982, IEEE Trans. Inf. Theory.

[34]  William Equitz,et al.  Successive refinement of information , 1991, IEEE Trans. Inf. Theory.

[35]  Anant Sahai,et al.  Equivalence perspectives in communication, source-channel connections and universal source-channel separation , 2009, ArXiv.

[36]  Alon Orlitsky,et al.  Worst-case interactive communication - II: Two messages are not optimal , 1991, IEEE Trans. Inf. Theory.

[37]  Thomas J. Goblick,et al.  Theoretical limitations on the transmission of data from analog sources , 1965, IEEE Trans. Inf. Theory.

[38]  Patrick Mitran On a Markov Lemma and Typical Sequences for Polish Alphabets , 2015, IEEE Transactions on Information Theory.

[39]  Suhas N. Diggavi,et al.  Approximating the Gaussian Multiple Description Rate Region Under Symmetric Distortion Constraints , 2009, IEEE Transactions on Information Theory.

[40]  Prakash Ishwar,et al.  Infinite-message distributed source coding for two-terminal interactive computing , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[41]  Suhas N. Diggavi,et al.  Multistage successive refinement for Wyner-Ziv source coding with degraded side informations , 2006, 2006 IEEE International Symposium on Information Theory.

[42]  Muriel Médard,et al.  A Theory of Network Equivalence— Part I: Point-to-Point Channels , 2011, IEEE Transactions on Information Theory.

[43]  Zhi-Quan Luo,et al.  Multiterminal Source–Channel Communication Over an Orthogonal Multiple-Access Channel , 2007, IEEE Transactions on Information Theory.

[44]  Neri Merhav,et al.  Two-way successively refined joint source-channel coding , 2006, IEEE Transactions on Information Theory.

[45]  Neri Merhav,et al.  On hierarchical joint source-channel coding with degraded side information , 2006, IEEE Transactions on Information Theory.

[46]  Raymond W. Yeung,et al.  Information Theory and Network Coding , 2008 .

[47]  Alon Orlitsky,et al.  Worst-case interactive communication I: Two messages are almost optimal , 1990, IEEE Trans. Inf. Theory.

[48]  Yasutada Oohama,et al.  The Rate-Distortion Function for the Quadratic Gaussian CEO Problem , 1998, IEEE Trans. Inf. Theory.

[49]  Prakash Ishwar,et al.  Infinite-message interactive function computation in collocated networks , 2010, 2010 IEEE International Symposium on Information Theory.

[50]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[51]  S. Borade Network information flow: limits and achievability , 2002, Proceedings IEEE International Symposium on Information Theory,.

[52]  Te Sun Han Multicasting correlated multi-source to multi-sink over a network , 2009 .

[53]  Lihua Song,et al.  A separation theorem for single-source network coding , 2006, IEEE Transactions on Information Theory.

[54]  Han-I Su,et al.  Cascade multiterminal source coding , 2009, 2009 IEEE International Symposium on Information Theory.

[55]  Thomas M. Cover,et al.  Network Information Theory , 2001 .