A Unified Framework for One-Shot Achievability via the Poisson Matching Lemma

We introduce a fundamental lemma called the Poisson matching lemma, and apply it to prove one-shot achievability results for various settings, namely channels with state information at the encoder, lossy source coding with side information at the decoder, joint source-channel coding, broadcast channels, distributed lossy source coding, multiple access channels and channel resolvability. Our one-shot bounds improve upon the best known one-shot bounds in most of the aforementioned settings (except multiple access channels and channel resolvability, where we recover bounds comparable to the best known bounds), with shorter proofs in some settings even when compared to the conventional asymptotic approach using typicality. The Poisson matching lemma replaces both the packing and covering lemmas, greatly simplifying the error analysis. This paper extends the work of Li and El Gamal on Poisson functional representation, which mainly considered variable-length source coding settings, whereas this paper studies fixed-length settings, and is not limited to source coding, showing that the Poisson functional representation is a viable alternative to typicality for most problems in network information theory.

[1]  Ayfer Özgür,et al.  Minimax Learning for Remote Prediction , 2018, 2018 IEEE International Symposium on Information Theory (ISIT).

[2]  Cheuk Ting Li,et al.  Pairwise Multi-marginal Optimal Transport and Embedding for Earth Mover's Distance. , 2019 .

[3]  Sergio Verdú,et al.  One-shot mutual covering lemma and Marton's inner bound with a common message , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[4]  Claude E. Shannon,et al.  Certain Results in Coding Theory for Noisy Channels , 1957, Inf. Control..

[5]  Aaron D. Wyner,et al.  The common information of two dependent random variables , 1975, IEEE Trans. Inf. Theory.

[6]  Jonathan Scarlett,et al.  On the Dispersions of the Gel’fand–Pinsker Channel and Dirty Paper Coding , 2013, IEEE Transactions on Information Theory.

[7]  Amiel Feinstein,et al.  A new basic theorem of information theory , 1954, Trans. IRE Prof. Group Inf. Theory.

[8]  Sergio Verdú,et al.  Approximation theory of output statistics , 1993, IEEE Trans. Inf. Theory.

[9]  Jaikumar Radhakrishnan,et al.  The Communication Complexity of Correlation , 2007, Twenty-Second Annual IEEE Conference on Computational Complexity (CCC'07).

[10]  Masahito Hayashi,et al.  Second order analysis for joint source-channel coding with Markovian source , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[11]  Vincent Y. F. Tan,et al.  Rényi Resolvability and Its Applications to the Wiretap Channel , 2017, IEEE Transactions on Information Theory.

[12]  Sergio Verdú,et al.  Rejection Sampling and Noncausal Sampling Under Moment Constraints , 2018, 2018 IEEE International Symposium on Information Theory (ISIT).

[13]  Rudolf Ahlswede,et al.  Multi-way communication channels , 1973 .

[14]  Aaron D. Wyner,et al.  The rate-distortion function for source coding with side information at the decoder , 1976, IEEE Trans. Inf. Theory.

[15]  Yury Polyanskiy,et al.  On dispersion of compound DMCs , 2013, 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[16]  H. Vincent Poor,et al.  Channel Coding Rate in the Finite Blocklength Regime , 2010, IEEE Transactions on Information Theory.

[17]  Paul W. Cuff,et al.  Distributed Channel Synthesis , 2012, IEEE Transactions on Information Theory.

[18]  Aaron D. Wyner,et al.  On source coding with side information at the decoder , 1975, IEEE Trans. Inf. Theory.

[19]  A. C. Berry.  The accuracy of the Gaussian approximation to the sum of independent variates , 1941 .

[20]  H. Vincent Poor,et al.  Channel coding: non-asymptotic fundamental limits , 2010 .

[21]  Abbas El Gamal,et al.  Strong functional representation lemma and applications to coding theorems , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[22]  Amin Gohari,et al.  Non-asymptotic output statistics of Random Binning and its applications , 2013, 2013 IEEE International Symposium on Information Theory.

[23]  Neri Merhav,et al.  Exact random coding secrecy exponents for the wiretap channel , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[24]  Aaron D. Wyner,et al.  The rate-distortion function for source coding with side information at the decoder , 1976, IEEE Trans. Inf. Theory.

[25]  Robert G. Gallager,et al.  A simple derivation of the coding theorem and some applications , 1965, IEEE Trans. Inf. Theory.

[26]  Sergio Verdú,et al.  Fixed-Length Lossy Compression in the Finite Blocklength Regime , 2011, IEEE Transactions on Information Theory.

[27]  Yingbin Liang,et al.  Rate Regions for Relay Broadcast Channels , 2006, IEEE Transactions on Information Theory.

[28]  Éva Tardos,et al.  Approximation algorithms for classification problems with pairwise relationships: metric labeling and Markov random fields , 2002, JACM.

[29]  Cheuk Ting Li,et al.  A Unified Framework for One-shot Achievability via the Poisson Matching Lemma , 2018, 2019 IEEE International Symposium on Information Theory (ISIT).

[30]  Paul W. Cuff,et al.  Exact Soft-Covering Exponent , 2018, 2018 IEEE International Symposium on Information Theory (ISIT).

[31]  H. Vincent Poor,et al.  The Likelihood Encoder for Lossy Compression , 2014, IEEE Transactions on Information Theory.

[32]  A. D. Wyner,et al.  The wire-tap channel , 1975, The Bell System Technical Journal.

[33]  Patrick P. Bergmans,et al.  Random coding theorem for broadcast channels with degraded components , 1973, IEEE Trans. Inf. Theory.

[34]  Sergio Verdú,et al.  Lossy Joint Source-Channel Coding in the Finite Blocklength Regime , 2012, IEEE Transactions on Information Theory.

[35]  Andrei Z. Broder,et al.  On the resemblance and containment of documents , 1997, Proceedings. Compression and Complexity of SEQUENCES 1997 (Cat. No.97TB100171).

[36]  Robert G. Gallager,et al.  Capacity and coding for degraded broadcast channels , 1974 .

[37]  H. Vincent Poor,et al.  On the Equivalence of Two Achievable Regions for the Broadcast Channel , 2011, IEEE Transactions on Information Theory.

[38]  J. Wolfowitz.  Simultaneous channels , 1959 .

[39]  Sergio Verdú,et al.  Sharp Bounds for Mutual Covering , 2019, IEEE Transactions on Information Theory.

[40]  Masahito Hayashi,et al.  General nonasymptotic and asymptotic formulas in channel resolvability and identification capacity and their application to the wiretap channel , 2006, IEEE Transactions on Information Theory.

[41]  Moses Charikar,et al.  Similarity estimation techniques from rounding algorithms , 2002, STOC '02.

[42]  Rudolf Ahlswede,et al.  Source coding with side information and a converse for degraded broadcast channels , 1975, IEEE Trans. Inf. Theory.

[43]  Amin Gohari,et al.  A technique for deriving one-shot achievability results in network information theory , 2013, 2013 IEEE International Symposium on Information Theory.

[44]  R. Ahlswede.  The Capacity Region of a Channel with Two Senders and Two Receivers , 1974 .

[45]  Katalin Marton,et al.  A coding theorem for the discrete memoryless broadcast channel , 1979, IEEE Trans. Inf. Theory.

[46]  Mark Braverman,et al.  Public vs Private Coin in Bounded-Round Information , 2014, ICALP.

[47]  Jingbo Liu,et al.  Dispersion Bound for the Wyner-Ahlswede-Körner Network via Reverse Hypercontractivity on Types , 2018, 2018 IEEE International Symposium on Information Theory (ISIT).

[48]  Shlomo Shamai,et al.  On information rates for mismatched decoders , 1994, IEEE Trans. Inf. Theory.

[49]  Vincent Yan Fu Tan,et al.  Nonasymptotic and Second-Order Achievability Bounds for Coding With Side-Information , 2013, IEEE Transactions on Information Theory.

[50]  Mohammad Hossein Yassaee.  Almost Exact Analysis of Soft Covering Lemma via Large Deviation , 2019, 2019 IEEE International Symposium on Information Theory (ISIT).

[51]  Omer Angel,et al.  Pairwise optimal coupling of multiple random variables , 2019, 1903.00632.

[52]  Sergio Verdú,et al.  On α-decodability and α-likelihood decoder , 2017, 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[53]  Masahito Hayashi,et al.  General formulas for capacity of classical-quantum channels , 2003, IEEE Transactions on Information Theory.

[54]  Jacob Wolfowitz,et al.  Multiple Access Channels , 1978 .

[55]  V. Bentkus.  On the dependence of the Berry–Esseen bound on dimension , 2003 .

[56]  D. Blackwell,et al.  The Capacity of a Class of Channels , 1959 .

[57]  Sergio Verdú,et al.  Non-asymptotic achievability bounds in multiuser information theory , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).