Strong Converses are Just Edge Removal Properties

This paper explores the relationship between two ideas in the network information theory: edge removal and strong converses. Edge removal properties state that if an edge of small capacity is removed from a network, the capacity region does not change too much. Strong converses state that, for rates outside the capacity region, the probability of error converges to 1 as the blocklength goes to infinity. Various notions of edge removal and strong converse are defined, depending on how edge capacity and error probability scale with blocklength, and relations between them are proved. Each class of strong converse implies a specific class of edge removal. The opposite directions are proved for deterministic networks. Furthermore, a technique based on a novel, causal version of the blowing-up lemma is used to prove that for discrete memoryless networks, the weak edge removal property—that the capacity region changes continuously as the capacity of an edge vanishes—is equivalent to the exponentially strong converse—that outside the capacity region, the probability of error goes to 1 exponentially fast. This result is used to prove exponentially strong converses for several examples, including the discrete two-user interference channel with strong interference, with only a small variation from traditional weak converse proofs.

[1]  Yu Xiang,et al.  A few meta-theorems in network information theory , 2014, 2014 IEEE Information Theory Workshop (ITW 2014).

[2]  Abdel R. El Gamal,et al.  On information flow in relay networks , 1981 .

[3]  Gunter Dueck,et al.  Reliability function of a discrete memoryless channel at rates above capacity (Corresp.) , 1979, IEEE Trans. Inf. Theory.

[4]  Vincent Y. F. Tan,et al.  Strong Converse Theorems for Multimessage Networks with Tight Cut-Set Bound , 2019, Probl. Inf. Transm..

[5]  J. Wolfowitz.  The coding of messages subject to chance errors , 1957 .

[6]  Raymond W. Yeung,et al.  A framework for linear information inequalities , 1997, IEEE Trans. Inf. Theory.

[7]  Michael Langberg,et al.  Source coding for dependent sources , 2012, 2012 IEEE Information Theory Workshop.

[8]  Vincent Y. F. Tan,et al.  A Proof of the Strong Converse Theorem for Gaussian Multiple Access Channels , 2015, IEEE Transactions on Information Theory.

[9]  W. Marsden.  I and J , 2012 .

[10]  M. Effros,et al.  A continuity theory for lossless source coding over networks , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[11]  Igal Sason,et al.  Concentration of Measure Inequalities in Information Theory, Communications, and Coding , 2012, Found. Trends Commun. Inf. Theory.

[12]  Michael Langberg,et al.  Outer bounds and a functional study of the edge removal problem , 2013, 2013 IEEE Information Theory Workshop (ITW).

[13]  Yasutada Oohama.  Exponent function for asymmetric broadcast channels at rates outside the capacity region , 2016, 2016 International Symposium on Information Theory and Its Applications (ISITA).

[14]  Andreas J. Winter,et al.  Coding theorem and strong converse for quantum channels , 1999, IEEE Trans. Inf. Theory.

[15]  Vincent Y. F. Tan,et al.  Strong converse theorems for classes of multimessage multicast networks: A Rényi divergence approach , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[16]  D. A. Edwards.  On the existence of probability measures with given marginals , 1978 .

[17]  Katalin Marton,et al.  A simple proof of the blowing-up lemma , 1986, IEEE Trans. Inf. Theory.

[18]  Muriel Médard,et al.  A Theory of Network Equivalence— Part I: Point-to-Point Channels , 2011, IEEE Transactions on Information Theory.

[19]  R. Ahlswede.  An elementary proof of the strong converse theorem for the multiple-access channel , 1982 .

[20]  Michael Langberg,et al.  Can negligible cooperation increase network reliability? , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[21]  Suguru Arimoto,et al.  On the converse to the coding theorem for discrete memoryless channels (Corresp.) , 1973, IEEE Trans. Inf. Theory.

[22]  Alex J. Grant,et al.  Network Coding Capacity Regions via Entropy Functions , 2012, IEEE Transactions on Information Theory.

[23]  Yasutada Oohama.  Strong converse exponent for degraded broadcast channels at rates outside the capacity region , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[24]  Hiroshi Sato,et al.  The capacity of the Gaussian interference channel under strong interference , 1981, IEEE Trans. Inf. Theory.

[25]  Michael Langberg,et al.  The benefit of encoder cooperation in the presence of state information , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[26]  Abbas El Gamal,et al.  Capacity theorems for the relay channel , 1979, IEEE Trans. Inf. Theory.

[27]  Rudolf Ahlswede,et al.  Common Randomness in Information Theory and Cryptography - Part II: CR Capacity , 1998, IEEE Trans. Inf. Theory.

[28]  Michael Langberg,et al.  On the cost and benefit of cooperation , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[29]  P. Gács,et al.  Bounds on conditional probabilities with applications in multi-user communication , 1976 .

[30]  D. A. Bell,et al.  Information Theory and Reliable Communication , 1969 .

[31]  Vincent Y. F. Tan,et al.  Strong converse theorems for discrete memoryless networks with tight cut-set bound , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[32]  Mehul Motani,et al.  A Case Where Interference Does Not Affect the Channel Dispersion , 2014, IEEE Transactions on Information Theory.

[33]  V. Anantharam,et al.  Generalized Network Sharing Outer Bound and the Two-Unicast Problem , 2011, 2011 International Symposium on Networking Coding.

[34]  Yasutada Oohama.  Exponent function for one helper source coding problem at rates outside the rate region , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[35]  Michael Langberg,et al.  The unbounded benefit of encoder cooperation for the k-user MAC , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[36]  Lizhong Zheng,et al.  Euclidean Information Theory , 2008, 2008 IEEE International Zurich Seminar on Communications.

[37]  Tracey Ho,et al.  On the impact of a single edge on the network coding capacity , 2011, 2011 Information Theory and Applications Workshop.

[38]  Tomohiro Ogawa,et al.  Strong converse to the quantum channel coding theorem , 1999, IEEE Trans. Inf. Theory.

[39]  Yasutada Oohama.  Strong converse theorems for degraded broadcast channels with feedback , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[40]  Tracey Ho,et al.  On equivalence between network topologies , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[41]  Sergio Verdú,et al.  Smoothing Brascamp-Lieb inequalities and strong converses for common randomness generation , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[42]  Thomas M. Cover,et al.  Network Information Theory , 2001 .

[43]  Michael Langberg,et al.  On the Capacity Advantage of a Single Bit , 2016, 2016 IEEE Globecom Workshops (GC Wkshps).

[44]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[45]  Michael Langberg,et al.  On a capacity equivalence between network and index coding and the edge removal problem , 2013, 2013 IEEE International Symposium on Information Theory.

[46]  M. Raginsky.  Concentration of Measure Inequalities in Information Theory, Communications, and Coding: Second Edition , 2014 .

[47]  Mohammad Reza Aref,et al.  The capacity of the semideterministic relay channel , 1982, IEEE Trans. Inf. Theory.

[48]  Suhas N. Diggavi,et al.  Wireless Network Information Flow: A Deterministic Approach , 2009, IEEE Transactions on Information Theory.

[49]  Jörg Kliewer,et al.  Equivalence for networks with adversarial state , 2014, 2014 IEEE International Symposium on Information Theory.

[50]  Yuan Zhou.  Introduction to Coding Theory , 2010 .

[51]  Yasutada Oohama.  Exponent function for Wyner-Ziv source coding problem at rates below the rate distortion function , 2016, 2016 International Symposium on Information Theory and Its Applications (ISITA).

[52]  Michael Langberg,et al.  Can Negligible Rate Increase Network Reliability? , 2018, IEEE Transactions on Information Theory.

[53]  Max H. M. Costa,et al.  The capacity region of the discrete memoryless interference channel with strong interference , 1987, IEEE Trans. Inf. Theory.

[54]  WeiHsin Gu.  On Achievable Rate Regions for Source Coding over Networks , 2009 .

[55]  H. Vincent Poor,et al.  Channel Coding Rate in the Finite Blocklength Regime , 2010, IEEE Transactions on Information Theory.

[56]  Michael Langberg,et al.  Network coding: Is zero error always possible? , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[57]  Tracey Ho,et al.  On the power of cooperation: Can a little help a lot? , 2014, 2014 IEEE International Symposium on Information Theory.