Concentration of Measure Inequalities in Information Theory, Communications, and Coding

Concentration inequalities have been the subject of exciting developments during the last two decades, and have been intensively studied and used as a powerful tool in various areas. These include convex geometry, functional analysis, statistical physics, mathematical statistics, pure and applied probability theory (e.g., concentration of measure phenomena in random graphs, random matrices, and percolation), information theory, theoretical computer science, learning theory, and dynamical systems.This monograph focuses on some of the key modern mathematical tools that are used for the derivation of concentration inequalities, on their links to information theory, and on their various applications to communications and coding. In addition to being a survey, this monograph also includes various new recent results derived by the authors.The first part of the monograph introduces classical concentration inequalities for martingales, aswell as some recent refinements and extensions. The power and versatility of the martingale approach is exemplified in the context of codes defined on graphs and iterative decoding algorithms, as well as codes for wireless communication.The second part of the monograph introduces the entropy method, an information-theoretic technique for deriving concentration inequalities for functions of many independent random variables. The basic ingredients of the entropy method are discussed first in conjunction with the closely related topic of logarithmic Sobolev inequalities, which are typical of the so-called functional approach to studying the concentration of measure phenomenon. The discussion on logarithmic Sobolev inequalities is complemented by a related viewpoint based on probability in metric spaces. This viewpoint centers around the so-called transportation-cost inequalities, whose roots are in information theory. Some representative results on concentration for dependent random variables are briefly summarized, with emphasis on their connections to the entropy method. Finally, we discuss several applications of the entropy method and related information-theoretic tools to problems in communications and coding. These include strong converses, empirical distributions of good channel codes with non-vanishing error probability, and an information-theoretic converse for concentration of measure.

[1]  B. Øksendal.  Stochastic differential equations : an introduction with applications , 1987 .

[2]  Djalil CHAFAÏ.  Entropies, convexity, and functional inequalities , 2002 .

[3]  Igal Sason.  Improved lower bounds on the total variation distance and relative entropy for the Poisson approximation , 2013, 2013 Information Theory and Applications Workshop (ITA).

[4]  Lawrence K. Saul,et al.  Large Deviation Methods for Approximate Probabilistic Inference , 1998, UAI.

[5]  Nicholas Kalouptsidis,et al.  Achievable Rates for Nonlinear Volterra Channels , 2011, IEEE Transactions on Information Theory.

[6]  D. Berend,et al.  On the concentration of the missing mass , 2012, 1210.3248.

[7]  Abbas El Gamal,et al.  Network Information Theory , 2021, 2021 IEEE 3rd International Conference on Advanced Trends in Information Theory (ATIT).

[8]  Ioannis Kontoyiannis,et al.  Measure Concentration for Compound Poisson Distributions , 2005, Electronic Communications in Probability.

[9]  R. A. Doney,et al.  4. Probability and Random Processes , 1993 .

[10]  P. MassartLedoux,et al.  Concentration Inequalities Using the Entropy Method , 2002 .

[11]  A. Osękowski.  Weak type inequalities for conditionally symmetric martingales , 2010 .

[12]  Sean P. Meyn,et al.  Relative entropy and exponential deviation bounds for general Markov chains , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[13]  E. Ordentlich,et al.  Inequalities for the L1 Deviation of the Empirical Distribution , 2003 .

[14]  Daniel A. Spielman,et al.  Improved low-density parity-check codes using irregular graphs and belief propagation , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[15]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[16]  D. Spielman,et al.  Expander codes , 1996 .

[17]  Xiao Ma,et al.  Binary intersymbol interference channels: Gallager codes, density evolution, and code performance bounds , 2003, IEEE Transactions on Information Theory.

[18]  Sanjeev R. Kulkarni,et al.  Probability Estimation in the Rare-Events Regime , 2011, IEEE Transactions on Information Theory.

[19]  H. Vincent Poor,et al.  Channel Coding Rate in the Finite Blocklength Regime , 2010, IEEE Transactions on Information Theory.

[20]  K. Marton,et al.  Correction. Measure concentration for Euclidean distance in the case of dependent random variables. , 2010 .

[21]  L. Kantorovich.  On the Translocation of Masses , 2006 .

[22]  Igal Sason,et al.  Tightened Exponential Bounds for Discrete Time, Conditionally Symmetric Martingales with Bounded Jumps , 2012, ArXiv.

[23]  J. Steele.  An Efron-Stein inequality for nonsymmetric statistics , 1986 .

[24]  M. Talagrand.  Concentration of measure and isoperimetric inequalities in product spaces , 1994, math/9406212.

[25]  John Shawe-Taylor,et al.  PAC-Bayesian Inequalities for Martingales , 2011, IEEE Transactions on Information Theory.

[26]  Kuldeep Kumar.  Robust Statistics, 2nd edition by P.J. Huber & E.M. Ronchetti [book review] , 2011 .

[27]  P. Spreij.  Probability and Measure , 1996 .

[28]  Pawel Hitczenko,et al.  Beyond the method of bounded differences , 1997, Microsurveys in Discrete Probability.

[29]  Neri Merhav,et al.  Statistical Physics and Information Theory , 2010, Found. Trends Commun. Inf. Theory.

[30]  Sergio Verdú,et al.  Simulation of random processes and rate-distortion theory , 1996, IEEE Trans. Inf. Theory.

[31]  Igal Sason,et al.  An Improved Sphere-Packing Bound for Finite-Length Codes Over Symmetric Memoryless Channels , 2006, IEEE Transactions on Information Theory.

[32]  Sergio Verdú.  Mismatched estimation and relative entropy , 2010, IEEE Trans. Inf. Theory.

[33]  Marion Kee,et al.  Analysis , 2004, Machine Translation.

[34]  D. Freedman.  On Tail Probabilities for Martingales , 1975 .

[35]  Emmanuel Rio,et al.  On McDiarmid's concentration inequality , 2013 .

[36]  Emanuel Milman,et al.  Properties of isoperimetric, functional and Transport-Entropy inequalities via concentration , 2009, 0909.0207.

[37]  H. Jeffreys.  A Treatise on Probability , 1922, Nature.

[38]  Giuseppe Toscani,et al.  An information-theoretic proof of Nash's inequality , 2012, ArXiv.

[39]  R. Gray,et al.  A Generalization of Ornstein's $\bar d$ Distance with Applications to Information Theory , 1975 .

[40]  Dudley,et al.  Real Analysis and Probability: Measurability: Borel Isomorphism and Analytic Sets , 2002 .

[41]  A. Guillin,et al.  Transportation cost-information inequalities and applications to random dynamical systems and diffusions , 2004, math/0410172.

[42]  Amir Dembo,et al.  Information theoretic inequalities , 1991, IEEE Trans. Inf. Theory.

[43]  Udo Augustin,et al.  GedÄchtnisfreie KanÄle für diskrete Zeit , 1966 .

[44]  Aaron B. Wagner,et al.  Refinement of the Sphere-Packing Bound: Asymmetric Channels , 2012, IEEE Transactions on Information Theory.

[45]  Sergio Verdú,et al.  Empirical distribution of good channel codes with non-vanishing error probability (extended version) , 2013, ArXiv.

[46]  Aaron D. Wyner,et al.  A theorem on the entropy of certain binary sequences and applications-II , 1973, IEEE Trans. Inf. Theory.

[47]  E. Carlen.  Superadditivity of Fisher's information and logarithmic Sobolev inequalities , 1991 .

[48]  Subhankar Ghosh,et al.  Applications of size biased couplings for concentration of measures , 2011 .

[49]  Thomas M. Cover,et al.  Elements of information theory (2. ed.) , 2006 .

[50]  I. Csiszár.  Sanov Property, Generalized $I$-Projection and a Conditional Limit Theorem , 1984 .

[51]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[52]  M. Ledoux.  The concentration of measure phenomenon , 2001 .

[53]  S. Bobkov,et al.  On Modified Logarithmic Sobolev Inequalities for Bernoulli and Poisson Measures , 1998 .

[54]  Daniel Berend,et al.  A Reverse Pinsker Inequality , 2012, ArXiv.

[55]  J. Tropp.  FREEDMAN'S INEQUALITY FOR MATRIX MARTINGALES , 2011, 1101.3039.

[56]  Andrea Montanari,et al.  Tight bounds for LDPC and LDGM codes under MAP decoding , 2004, IEEE Transactions on Information Theory.

[57]  Igal Sason,et al.  An Information-Theoretic Perspective of the Poisson Approximation via the Chen-Stein Method , 2012, ArXiv.

[58]  Igal Sason.  On the concentration of the crest factor for OFDM signals , 2011, 2011 8th International Symposium on Wireless Communication Systems.

[59]  Andrew W. Swift,et al.  A refinement of Hoeffding's inequality , 2013 .

[60]  H. Chernoff.  A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations , 1952 .

[61]  Marco Breiling,et al.  A logarithmic upper bound on the minimum distance of turbo codes , 2004, IEEE Transactions on Information Theory.

[62]  Gerhard Wunder,et al.  Generalized bounds on the crest-factor distribution of OFDM signals with applications to code design , 2004, IEEE Transactions on Information Theory.

[63]  Werner Horsthemke.  Time's Arrow: The Origins of Thermodynamic Behavior, Michael C. Mackey. Springer, Boca Raton, FL (1992), $49.00 (cloth), 172 pp , 1994 .

[64]  Christos P. Kitsos,et al.  Logarithmic Sobolev Inequalities for Information Measures , 2009, IEEE Transactions on Information Theory.

[65]  N. Gozlan.  A characterization of dimension free concentration in terms of transportation inequalities , 2008, 0804.3089.

[66]  Larry Goldstein,et al.  Concentration inequalities via zero bias couplings , 2013, 1304.5001.

[67]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[68]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[69]  Colin McDiarmid,et al.  Centering Sequences with Bounded Differences , 1997, Combinatorics, Probability and Computing.

[70]  Sergio Verdú,et al.  Fixed-Length Lossy Compression in the Finite Blocklength Regime , 2011, IEEE Transactions on Information Theory.

[71]  Thomas M. Cover,et al.  Broadcast channels , 1972, IEEE Trans. Inf. Theory.

[72]  S. Shamai,et al.  The empirical distribution of good codes , 1995, Proceedings of 1995 IEEE International Symposium on Information Theory.

[73]  C. Villani,et al.  Generalization of an Inequality by Talagrand and Links with the Logarithmic Sobolev Inequality , 2000 .

[74]  Andreas F. Molisch,et al.  Wireless Communications , 2005 .

[75]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[76]  Holger Boche,et al.  Upper bounds on the statistical distribution of the crest-factor in OFDM transmission , 2003, IEEE Trans. Inf. Theory.

[77]  Ioannis Karatzas,et al.  Brownian Motion and Stochastic Calculus , 1987 .

[78]  Alain Glavieux,et al.  Iterative correction of intersymbol interference : turbo-equalization : Iterative and Turbo decoding , 1995 .

[79]  Katalin Marton.  Bounding relative entropy by the relative entropy of local specifications in product spaces , 2009 .

[80]  C. Villani.  Topics in Optimal Transportation , 2003 .

[81]  C. Villani.  Optimal Transport: Old and New , 2008 .

[82]  J. Rosenthal.  A First Look at Rigorous Probability Theory , 2000 .

[83]  J. Steele.  Probability theory and combinatorial optimization , 1987 .

[84]  Peter Harremoës,et al.  Rényi Divergence and Kullback-Leibler Divergence , 2012, IEEE Transactions on Information Theory.

[85]  Christian L'eonard,et al.  Transport Inequalities. A Survey , 2010, 1003.3852.

[86]  Max H. M. Costa,et al.  A new entropy power inequality , 1985, IEEE Trans. Inf. Theory.

[87]  S. Bobkov,et al.  Modified Logarithmic Sobolev Inequalities in Discrete Settings , 2006 .

[88]  Daniel A. Spielman,et al.  Efficient erasure correcting codes , 2001, IEEE Trans. Inf. Theory.

[89]  L. M. M.-T..  Theory of Probability , 1929, Nature.

[90]  Andrea Montanari,et al.  On the concentration of the number of solutions of random satisfiability formulas , 2010, Random Struct. Algorithms.

[91]  Djalil CHAFAÏ.  Entropies, convexity, and functional inequalities : On Phi-entropies and Phi-Sobolev inequalities , 2004 .

[92]  R. Dobrushin,et al.  Completely Analytical Gibbs Fields , 1985 .

[93]  G. Lugosi.  Concentration-of-measure inequalities Lecture notes by Gábor Lugosi , 2005 .

[94]  Igal Sason,et al.  On Universal Properties of Capacity-Approaching LDPC Code Ensembles , 2007, IEEE Transactions on Information Theory.

[95]  M. Shokrollahi,et al.  Capacity-achieving sequences , 2001 .

[96]  Nicolas Macris,et al.  Concentration of magnetization for linear block codes , 2008, 2008 IEEE International Symposium on Information Theory.

[97]  Fan Chung Graham,et al.  Concentration Inequalities and Martingale Inequalities: A Survey , 2006, Internet Math..

[98]  William Matthews,et al.  A Linear Program for the Finite Block Length Converse of Polyanskiy–Poor–Verdú Via Nonsignaling Codes , 2011, IEEE Transactions on Information Theory.

[99]  A. Barron,et al.  Fisher information inequalities and the central limit theorem , 2001, math/0111020.

[100]  S. Chatterjee.  Concentration inequalities with exchangeable pairs (Ph.D. thesis) , 2005, math/0507526.

[101]  P. Federbush.  Partially Alternate Derivation of a Result of Nelson , 1969 .

[102]  M. Gromov.  Metric Structures for Riemannian and Non-Riemannian Spaces , 1999 .

[103]  Kazuoki Azuma.  WEIGHTED SUMS OF CERTAIN DEPENDENT RANDOM VARIABLES , 1967 .

[104]  Andrea Montanari,et al.  Further results on finite-length scaling for iteratively decoded LDPC ensembles , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[105]  G. Grimmett,et al.  Probability and random processes , 2002 .

[106]  Sergio Benedetto,et al.  Principles of Digital Transmission: With Wireless Applications , 1999 .

[107]  Joel A. Tropp,et al.  User-Friendly Tail Bounds for Sums of Random Matrices , 2010, Found. Comput. Math..

[108]  Rudolf Ahlswede,et al.  Source coding with side information and a converse for degraded broadcast channels , 1975, IEEE Trans. Inf. Theory.

[109]  M. Ledoux.  Concentration of measure and logarithmic Sobolev inequalities , 1999 .

[110]  Daniel Paulin,et al.  The convex distance inequality for dependent random variables, with applications to the stochastic travelling salesman and other problems , 2012, 1212.2014.

[111]  Andrea Montanari,et al.  Finite-Length Scaling for Iteratively Decoded LDPC Ensembles , 2004, IEEE Transactions on Information Theory.

[112]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[113]  L. Gross.  LOGARITHMIC SOBOLEV INEQUALITIES. , 1975 .

[114]  A. Dembo,et al.  TRANSPORTATION APPROACH TO SOME CONCEN- TRATION INEQUALITIES IN PRODUCT SPACES , 1996 .

[115]  T. Cover,et al.  IEEE TRANSACTIONSON INFORMATIONTHEORY,VOL. IT-30,N0. 6,NOVEmER1984 Correspondence On the Similarity of the Entropy Power Inequality The preceeding equations allow the entropy power inequality and the Brunn-Minkowski Inequality to be rewritten in the equiv , 2022 .

[116]  Imre Csiszár,et al.  Information Theory and Statistics: A Tutorial , 2004, Found. Trends Commun. Inf. Theory.

[117]  Quansheng Liu,et al.  Large deviation exponential inequalities for supermartingales , 2011 .

[118]  Nicolas Macris,et al.  On the concentration of the capacity for a code division multiple access system , 2007, 2007 IEEE International Symposium on Information Theory.

[119]  K. Marton.  Bounding $\bar{d}$-distance by informational divergence: a method to prove measure concentration , 1996 .

[120]  Nicolas Macris,et al.  Tight Bounds on the Capacity of Binary Input Random CDMA Systems , 2008, IEEE Transactions on Information Theory.

[121]  Nicolas Macris,et al.  Sharp Bounds for Optimal Decoding of Low-Density Parity-Check Codes , 2008, IEEE Transactions on Information Theory.

[122]  Donald Babbitt,et al.  An Initiation to Logarithmic Sobolev Inequalities , 2007 .

[123]  A. J. Stam.  Some Inequalities Satisfied by the Quantities of Information of Fisher and Shannon , 1959, Inf. Control..

[124]  Alice Guionnet,et al.  Lectures on Logarithmic Sobolev Inequalities , 2003 .

[125]  Barry Simon,et al.  Ultracontractivity and the Heat Kernel for Schrijdinger Operators and Dirichlet Laplacians , 1987 .

[126]  Igal Sason,et al.  Improved Lower Bounds on the Total Variation Distance for the Poisson Approximation , 2013, 1301.7504.

[127]  Sergio Verdú,et al.  Lossless Data Compression at Finite Blocklengths , 2012, ArXiv.

[128]  J. Kemperman.  On the Shannon capacity of an arbitrary channel , 1974 .

[129]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[130]  Kuldeep Kumar,et al.  Robust Statistics, 2nd edn , 2011 .

[131]  G. Bennett.  Probability Inequalities for the Sum of Independent Random Variables , 1962 .

[132]  Sergey G. Bobkov,et al.  A functional form of the isoperimetric inequality for the Gaussian measure , 1996 .

[133]  Dario Cordero-Erausquin,et al.  Some Applications of Mass Transport to Gaussian-Type Inequalities , 2002 .

[134]  Kenta Kasai,et al.  Analytical Solution of Covariance Evolution for Irregular LDPC Codes , 2009, IEEE Transactions on Information Theory.

[135]  Igal Sason,et al.  On concentration of measures for LDPC code ensembles , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[136]  Harry L. Van Trees,et al.  Detection, Estimation, and Modulation Theory, Part I , 1968 .

[137]  Sergio Verdu,et al.  Multiuser Detection , 1998 .

[138]  M. Talagrand.  A new look at independence , 1996 .

[139]  Rüdiger L. Urbanke,et al.  Modern Coding Theory , 2008 .

[140]  V. Peña.  A General Class of Exponential Inequalities for Martingales and Ratios , 1999 .

[141]  Ioannis Kontoyiannis,et al.  Sphere-covering, measure concentration, and source coding , 2001, IEEE Trans. Inf. Theory.

[142]  Sergey G. Bobkov,et al.  The Entropy Per Coordinate of a Random Vector is Highly Constrained Under Convexity Conditions , 2010, IEEE Transactions on Information Theory.

[143]  Gang Wang.  Sharp maximal inequalities for conditionally symmetric martingales and Brownian motion , 1991 .

[144]  K. Marton,et al.  An inequality for relative entropy and logarithmic Sobolev inequalities in Euclidean spaces , 2012 .

[145]  M. Talagrand.  Transportation cost for Gaussian and other product measures , 1996 .

[146]  Xiequan Fan,et al.  Hoeffding’s inequality for supermartingales , 2011, 1109.4359.

[147]  Robert G. Gallager,et al.  Capacity and coding for degraded broadcast channels , 1974 .

[148]  Daniel Berend,et al.  Minimum KL-Divergence on Complements of $L_{1}$ Balls , 2012, IEEE Transactions on Information Theory.

[149]  Katalin Marton,et al.  A simple proof of the blowing-up lemma , 1986, IEEE Trans. Inf. Theory.

[150]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[151]  Andrea Montanari,et al.  Maxwell Construction: The Hidden Bridge Between Iterative and Maximum a Posteriori Decoding , 2005, IEEE Transactions on Information Theory.

[152]  A. Dembo.  Information inequalities and concentration of measure , 1997 .

[153]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[154]  K. Marton.  A measure concentration inequality for contracting markov chains , 1996 .

[155]  M. Ledoux.  On Talagrand's deviation inequalities for product measures , 1997 .

[156]  Sergio Verdú,et al.  Optimal Lossless Data Compression: Non-Asymptotics and Asymptotics , 2014, IEEE Transactions on Information Theory.

[157]  T. Etzion,et al.  Which codes have cycle-free Tanner graphs? , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[158]  Shlomo Shamai,et al.  Performance Analysis of Linear Codes under Maximum-Likelihood Decoding: A Tutorial , 2006, Found. Trends Commun. Inf. Theory.

[159]  K. Dzhaparidze,et al.  On Bernstein-type inequalities for martingales , 2001 .

[160]  M. Talagrand.  Mean Field Models for Spin Glasses , 2011 .

[161]  Igal Sason,et al.  Refined bounds on the empirical distribution of good channel codes via concentration inequalities , 2013, 2013 IEEE International Symposium on Information Theory.

[162]  Alexander Barg,et al.  Random codes: Minimum distances and error exponents , 2002, IEEE Trans. Inf. Theory.

[163]  David L. Neuhoff,et al.  Process definitions of distortion-rate functions and source coding theorems , 1975, IEEE Trans. Inf. Theory.

[164]  C. McDiarmid.  Concentration , 1862, The Dental register.

[165]  Rudolf Ahlswede,et al.  Every bad code has a good subcode: A local converse to the coding theorem , 1976 .

[166]  S. Bobkov,et al.  Exponential Integrability and Transportation Cost Related to Logarithmic Sobolev Inequalities , 1999 .

[167]  Gordon Blower.  Random Matrices: High Dimensional Phenomena: Contents , 2009 .

[168]  P. Gács,et al.  Bounds on conditional probabilities with applications in multi-user communication , 1976 .

[169]  Michael I. Jordan,et al.  Matrix concentration inequalities via the method of exchangeable pairs , 2012, 1201.6002.

[170]  G. Crooks.  On Measures of Entropy and Information , 2015 .

[171]  K. Marton.  Measure concentration for Euclidean distance in the case of dependent random variables , 2004, math/0410168.

[172]  Joel H. Spencer,et al.  Sharp concentration of the chromatic number on random graphsGn, p , 1987, Comb..

[173]  S. Kudekar.  Statistical physics methods for sparse graph codes , 2009 .

[174]  Oliver Johnson,et al.  Entropy and the law of small numbers , 2005, IEEE Transactions on Information Theory.

[175]  R. Ahlswede.  An elementary proof of the strong converse theorem for the multiple-access channel , 1982 .

[176]  Cédric Villani,et al.  A short proof of the "Concavity of entropy power" , 2000, IEEE Trans. Inf. Theory.

[177]  Linyuan Lu,et al.  Complex Graphs and Networks (CBMS Regional Conference Series in Mathematics) , 2006 .

[178]  Paul-Marie Samson,et al.  Concentration of measure inequalities for Markov chains and $\Phi$-mixing processes , 2000 .

[179]  L. Evans.  Measure theory and fine properties of functions , 1992 .

[180]  Tsachy Weissman,et al.  The Information Lost in Erasures , 2008, IEEE Transactions on Information Theory.

[181]  Erik Ordentlich,et al.  A distribution dependent refinement of Pinsker's inequality , 2005, IEEE Transactions on Information Theory.

[182]  Neil Genzlinger.  A. and Q , 2006 .

[183]  S. Chatterjee.  Concentration Inequalities With Exchangeable Pairs , 2005 .

[184]  Nicholas Kalouptsidis,et al.  On the random coding exponent of nonlinear gaussian channels , 2009, 2009 IEEE Information Theory Workshop on Networking and Information Theory.

[185]  Holger Boche,et al.  The PAPR Problem in OFDM Transmission: New Directions for a Long-Lasting Problem , 2012, IEEE Signal Processing Magazine.

[186]  Sergio Verdú,et al.  Approximation theory of output statistics , 1993, IEEE Trans. Inf. Theory.

[187]  S. Chatterjee,et al.  Applications of Stein's method for concentration inequalities , 2009, 0906.1034.

[188]  Richard Mateosian,et al.  Old and New , 2006, IEEE Micro.

[189]  F. Willems.  The maximal-error and average-error capacity region of the broadcast channel are identical : A direct proof , 1990 .

[190]  B. Efron,et al.  The Jackknife Estimate of Variance , 1981 .

[191]  S. Bobkov,et al.  Concentration of the information in data with log-concave distributions , 2010, 1012.5457.

[192]  A. Rényi.  On Measures of Entropy and Information , 1961 .

[193]  S. Chatterjee.  Stein’s method for concentration inequalities , 2006, math/0604352.

[194]  F. Clarke,et al.  GROSS'S LOGARITHMIC SOBOLEV INEQUALITY: A SIMPLE PROOF , 1979 .

[195]  F. Chung,et al.  Complex Graphs and Networks , 2006 .

[196]  Young-Han Kim,et al.  State Amplification , 2008, IEEE Transactions on Information Theory.

[197]  Patrick P. Bergmans,et al.  Random coding theorem for broadcast channels with degraded components , 1973, IEEE Trans. Inf. Theory.

[198]  Nathan Ross.  Fundamentals of Stein's method , 2011, 1109.1880.

[199]  Shlomo Shamai,et al.  Variations on the Gallager bounds, connections, and applications , 2002, IEEE Trans. Inf. Theory.

[200]  O. Johnson.  Information Theory And The Central Limit Theorem , 2004 .

[201]  Nicholas Kalouptsidis,et al.  New achievable rates for nonlinear Volterra channels via martingale inequalities , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[202]  Andreas Maurer,et al.  Thermodynamics and Concentration , 2012, 1205.1595.

[203]  Patrick Cattiaux,et al.  On quadratic transportation cost inequalities , 2006 .

[204]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[205]  J. Lynch,et al.  A weak convergence approach to the theory of large deviations , 1997 .

[206]  R. Salem,et al.  Some properties of trigonometric series whose terms have random signs , 1954 .

[207]  Colin McDiarmid,et al.  Surveys in Combinatorics, 1989: On the method of bounded differences , 1989 .

[208]  Xiequan Fan,et al.  The missing factor in Bennett's inequality , 2012 .