Concentration of measure without independence: a unified approach via the martingale method

The concentration of measure phenomenon may be summarized as follows: a function of many weakly dependent random variables that is not too sensitive to any of its individual arguments will tend to take values very close to its expectation. This phenomenon is most completely understood when the arguments are mutually independent random variables, and there exist several powerful complementary methods for proving concentration inequalities, such as the martingale method, the entropy method, and the method of transportation inequalities. The setting of dependent arguments is much less well understood. This chapter focuses on the martingale method for deriving concentration inequalities without independence assumptions. In particular, we use the machinery of so-called Wasserstein matrices to show that the Azuma-Hoeffding concentration inequality for martingales with almost surely bounded differences, when applied in a sufficiently abstract setting, is powerful enough to recover and sharpen several known concentration results for nonproduct measures. Wasserstein matrices provide a natural formalism for capturing the interplay between the metric and the probabilistic structures, which is fundamental to the concentration phenomenon.

[1]  K. Marton.  Measure concentration for Euclidean distance in the case of dependent random variables , 2004, math/0410168.

[2]  A. Guillin,et al.  Transportation cost-information inequalities and applications to random dynamical systems and diffusions , 2004, math/0410172.

[3]  Hans Föllmer,et al.  Tail structure of markov chains on infinite product spaces , 1979 .

[4]  C. Külske.  Concentration Inequalities for Functions of Gibbs Fields with Application to Diffraction and Random Gibbs Measures , 2003 .

[5]  J. Lafferty,et al.  Measure concentration of strongly mixing processes with applications , 2007 .

[6]  R. Handel,et al.  Comparison Theorems for Gibbs Measures , 2013, 1308.4117.

[7]  E. Rio.  Inégalités de Hoeffding pour les fonctions lipschitziennes de suites dépendantes , 2000 .

[8]  Paul-Marie Samson,et al.  Concentration of measure inequalities for Markov chains and $\Phi$-mixing processes , 2000 .

[9]  K. Marton.  Measure concentration for a class of random processes , 1998 .

[10]  Aryeh Kontorovich,et al.  Uniform Chernoff and Dvoretzky-Kiefer-Wolfowitz-Type Inequalities for Markov Chains and Related Processes , 2012, J. Appl. Probab..

[11]  S. Chatterjee.  Concentration Inequalities With Exchangeable Pairs , 2005 .

[12]  L. Kontorovich.  Obtaining Measure Concentration from Markov Contraction , 2007, 0711.0987.

[13]  K. Marton.  Bounding $\bar{d}$-distance by informational divergence: a method to prove measure concentration , 1996 .

[14]  R. Dobrushin.  Prescribing a System of Random Variables by Conditional Distributions , 1970 .

[15]  Liming Wu,et al.  Poincaré and transportation inequalities for Gibbs measures under the Dobrushin uniqueness condition , 2006 .

[16]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[17]  H. Föllmer.  A covariance estimate for Gibbs measures , 1982 .

[18]  Olivier Wintenberger.  Weak transport inequalities and applications to exponential and oracle inequalities , 2015 .

[19]  P. Collet,et al.  Concentration inequalities for random fields via coupling , 2005, math/0503483.

[20]  WU L..  POINCARÉ AND TRANSPORTATION INEQUALITIES FOR GIBBS MEASURES UNDER THE DOBRUSHIN UNIQUENESS CONDITION , 2018 .

[21]  Prasad Tetali,et al.  Approximate tensorization of entropy at high temperature , 2014, 1405.0608.

[22]  R. Handel.  Probability in High Dimension , 2014 .

[23]  Solomon Kullback,et al.  Correction to A Lower Bound for Discrimination Information in Terms of Variation , 1970, IEEE Trans. Inf. Theory.

[24]  P. Levy,et al.  Problèmes concrets d'analyse fonctionnelle , 1952 .

[25]  D. Paulin.  Concentration inequalities for Markov chains by Marton couplings and spectral methods , 2012, 1212.2015.

[26]  K. Marton,et al.  An inequality for relative entropy and logarithmic Sobolev inequalities in Euclidean spaces , 2012 .

[27]  S. Kullback,et al.  A lower bound for discrimination information in terms of variation (Corresp.) , 1967, IEEE Trans. Inf. Theory.

[28]  Igal Sason,et al.  Concentration of Measure Inequalities in Information Theory, Communications, and Coding , 2012, Found. Trends Commun. Inf. Theory.

[29]  S. Chatterjee.  Stein’s method for concentration inequalities , 2006, math/0604352.

[30]  S. Bobkov,et al.  Exponential Integrability and Transportation Cost Related to Logarithmic Sobolev Inequalities , 1999 .

[31]  K. Marton.  Measure concentration and strong mixing , 2003 .

[32]  Dawei Hong,et al.  A Wavelet-Based Almost-Sure Uniform Approximation of Fractional Brownian Motion with a Parallel Algorithm , 2014, Journal of Applied Probability.

[33]  K. Ramanan,et al.  Concentration Inequalities for Dependent Random Variables via the Martingale Method , 2006, math/0609835.

[34]  Katalin Marton.  Logarithmic Sobolev inequalities in discrete product spaces: a proof by a transportation cost distance , 2015 .

[35]  S. Goldstein.  Maximal coupling , 1979 .