Minoration via Mixed Volumes and Cover's Problem for General Channels

We propose a method for establishing lower bounds on the supremum of processes in terms of packing numbers by means of mixedvolume inequalities (the Alexandrov-Fenchel inequality). A simple and general bound in terms of packing numbers under the convex distance is derived, from which some known bounds on the Gaussian processes and the Rademacher processes can be recovered when the convex set is taken to be the ball or the hypercube. However, the main thrust for our study of this approach is to handle non-i.i.d. (noncanonical) processes (correspondingly, the convex set is not a product set). As an application, we give a complete solution to an open problem of Thomas Cover in 1987 about the capacity of a relay channel in the general discrete memoryless setting.

[1]  T. Cover.  The Capacity of the Relay Channel , 1987 .

[2]  Ayfer Özgür,et al.  “The Capacity of the Relay Channel”: Solution to Cover’s Problem in the Gaussian Case , 2017, IEEE Transactions on Information Theory.

[3]  Gábor Lugosi,et al.  Concentration Inequalities , 2008, COLT.

[4]  R. Handel.  Probability in High Dimension , 2014 .

[5]  R. Schneider.  Convex Bodies: The Brunn–Minkowski Theory: Minkowski addition , 1993 .

[6]  Ayfer Özgür,et al.  Information Constrained Optimal Transport: From Talagrand, to Marton, to Cover , 2020, 2020 IEEE International Symposium on Information Theory (ISIT).

[7]  Ayfer Özgür,et al.  Improving on the Cut-Set Bound via Geometric Analysis of Typical Sets , 2016, IEEE Transactions on Information Theory.

[8]  B. Carl,et al.  Gelfand numbers of operators with values in a Hilbert space , 1988 .

[9]  Shahar Mendelson,et al.  Generalized dual Sudakov minoration via dimension-reduction—a program , 2016, Studia Mathematica.

[10]  Ayfer Özgür,et al.  Improving on the cut-set bound for general primitive relay channels , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[11]  N. Tomczak-Jaegermann.  Banach-Mazur distances and finite-dimensional operator ideals , 1989 .

[12]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[13]  P. Gács,et al.  Bounds on conditional probabilities with applications in multi-user communication , 1976 .

[14]  Q. Merigot,et al.  One more proof of the Alexandrov–Fenchel inequality , 2019, Comptes Rendus Mathematique.

[15]  M. Talagrand.  THE SUPREMUM OF SOME CANONICAL PROCESSES , 1994 .

[16]  Ayfer Özgür,et al.  Cut-set bound is loose for Gaussian relay networks , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[17]  V. Milman,et al.  Random subspaces of proportional dimension of finite dimensional normed spaces: Approach through the isoperimetric inequality , 1985 .

[18]  Jacob Wolfowitz,et al.  Notes on a General Strong Converse , 1968, Inf. Control..

[19]  V. V. Buldygin,et al.  Brunn-Minkowski inequality , 2000 .

[20]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[21]  Xianfu Wang.  Volumes of Generalized Unit Balls , 2005 .

[22]  David Haussler,et al.  A general minimax result for relative entropy , 1997, IEEE Trans. Inf. Theory.

[23]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[24]  Zhen Zhang,et al.  Partial converse for a relay channel , 1988, IEEE Trans. Inf. Theory.

[25]  O. Papaspiliopoulos.  High-Dimensional Probability: An Introduction with Applications in Data Science , 2020 .

[26]  Jingbo Liu.  Dispersion Bound for the Wyner-Ahlswede-Körner Network via a Semigroup Method on Types , 2021, IEEE Transactions on Information Theory.

[27]  P. Gács,et al.  Spreading of Sets in Product Spaces and Hypercontraction of the Markov Operator , 1976 .

[28]  Sergio Verdú,et al.  Second-Order Converses via Reverse Hypercontractivity , 2018, Mathematical Statistics and Learning.

[29]  Rafal Latala,et al.  Sudakov-type minoration for log-concave vectors , 2013, 1311.6428.

[30]  Jingbo Liu,et al.  Information Theory from A Functional Viewpoint , 2018 .

[31]  Richard M. Dudley,et al.  Sample Functions of the Gaussian Process , 1973 .

[32]  Roman Vershynin,et al.  High-Dimensional Probability , 2018 .

[33]  Peter L. Bartlett,et al.  Rademacher and Gaussian Complexities: Risk Bounds and Structural Results , 2003, J. Mach. Learn. Res..

[34]  Ramon van Handel,et al.  Mixed volumes and the Bochner method , 2018, Proceedings of the American Mathematical Society.

[35]  V. Milman,et al.  Almost Euclidean quotient spaces of subspaces of a finite-dimensional normed space , 1985 .

[36]  Young-Han Kim,et al.  Coding Techniques for Primitive Relay Channels , 2008 .

[37]  S. Geer.  Applications of empirical process theory , 2000 .

[38]  Ayfer Özgür,et al.  A solution to cover's problem for the binary symmetric relay channel: Geometry of sets on the hamming sphere , 2017, 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[39]  K. Böröczky,et al.  Covering the Sphere by Equal Spherical Balls , 2003 .

[40]  Ayfer Özgür,et al.  Capacity Upper Bounds for the Relay Channel via Reverse Hypercontractivity , 2018, IEEE Transactions on Information Theory.