The Multivariate Covering Lemma and its Converse

The multivariate covering lemma states that given a collection of $k$ codebooks, each of sufficiently large cardinality and independently generated according to one of the marginals of a joint distribution, one can always choose one codeword from each codebook such that the resulting $k$-tuple of codewords is jointly typical with respect to the joint distribution. We give a proof of this lemma for weakly typical sets. This allows achievability proofs that rely on the covering lemma to go through for continuous channels (e.g., Gaussian) without the need for quantization. The covering lemma and its converse are widely used in information theory, including in rate-distortion theory and in achievability results for multi-user channels.

[1]  J. Norris.  Appendix: probability and measure , 1997 .

[2]  Muriel Médard,et al.  A Theory of Network Equivalence – Part II: Multiterminal Channels , 2014, IEEE Transactions on Information Theory.

[3]  Thomas M. Cover,et al.  Network Information Theory , 2001 .

[4]  Katalin Marton,et al.  A coding theorem for the discrete memoryless broadcast channel , 1979, IEEE Trans. Inf. Theory.

[5]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[6]  Sergio Verdú.  Non-asymptotic covering lemmas , 2015, 2015 IEEE Information Theory Workshop (ITW).

[7]  Sergio Verdú,et al.  One-shot mutual covering lemma and Marton's inner bound with a common message , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[8]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[9]  Edward C. van der Meulen,et al.  A proof of Marton's coding theorem for the discrete memoryless broadcast channel , 1981, IEEE Trans. Inf. Theory.