Concentration of Measure for the Analysis of Randomized Algorithms

Randomized algorithms have become a central part of the algorithms curriculum based on their increasingly widespread use in modern applications. This book presents a coherent and unified treatment of probabilistic techniques for obtaining high- probability estimates on the performance of randomized algorithms. It covers the basic tool kit from the Chernoff-Hoeffding (CH) bounds to more sophisticated techniques like Martingales and isoperimetric inequalities, as well as some recent developments like Talagrand's inequality, transportation cost inequalities, and log-Sobolev inequalities. Along the way, variations on the basic theme are examined, such as CH bounds in dependent settings. The authors emphasize comparative study of the different methods, highlighting respective strengths and weaknesses in concrete example applications. The exposition is tailored to discrete settings sufficient for the analysis of algorithms, avoiding unnecessary measure-theoretic details, thus making the book accessible to computer scientists as well as probabilists and discrete mathematicians.

[1]  H. Chernoff.  A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations , 1952 .

[2]  D. Blackwell.  An analog of the minimax theorem for vector payoffs. , 1956 .

[3]  W. Hoeffding.  Probability inequalities for sum of bounded random variables , 1963 .

[4]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[5]  Béla Bollobás.  Chromatic number, girth and maximal degree , 1978, Discret. Math..

[6]  Vasek Chvátal,et al.  The tail of the hypergeometric distribution , 1979, Discret. Math..

[7]  G. Grimmett,et al.  Probability and random processes , 2002 .

[8]  Gary L. Miller,et al.  Parallel tree contraction and its application , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[9]  Michael Luby.  A Simple Parallel Algorithm for the Maximal Independent Set Problem , 1986, SIAM J. Comput..

[10]  J. Steele.  Probability theory and combinatorial optimization , 1987 .

[11]  Colin McDiarmid,et al.  Surveys in Combinatorics, 1989: On the method of bounded differences , 1989 .

[12]  William Pugh,et al.  Skip Lists: A Probabilistic Alternative to Balanced Trees , 1989, WADS.

[13]  Béla Bollobás,et al.  Compressions and isoperimetric inequalities , 1990, J. Comb. Theory, Ser. A.

[14]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[15]  Dexter Kozen,et al.  The Design and Analysis of Algorithms , 1991, Texts and Monographs in Computer Science.

[16]  Aravind Srinivasan,et al.  Chernoff-Hoeffding bounds for applications with limited independence , 1995, SODA '93.

[17]  Mihir Bellare,et al.  Randomness-efficient oblivious sampling , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[18]  Noam Nisan,et al.  Hardness vs Randomness , 1994, J. Comput. Syst. Sci..

[19]  M. Talagrand.  Concentration of measure and isoperimetric inequalities in product spaces , 1994, math/9406212.

[20]  Richard M. Karp,et al.  Probabilistic recurrence relations , 1994, JACM.

[21]  Jeong Han Kim.  On Brooks' Theorem for Sparse Graphs , 1995, Comb. Probab. Comput..

[22]  R. B. Hayward,et al.  Large Deviations for Quicksort , 1996, J. Algorithms.

[23]  Rajeev Motwani,et al.  Randomized algorithms , 1996, CSUR.

[24]  K. Marton.  Bounding $\bar{d}$-distance by informational divergence: a method to prove measure concentration , 1996 .

[25]  Colin McDiarmid,et al.  Centering Sequences with Bounded Differences , 1997, Combinatorics, Probability and Computing.

[26]  Alessandro Panconesi,et al.  Nearly optimal distributed edge colouring in O(log log n) rounds , 1997, SODA '97.

[27]  Jeong Han Kim,et al.  Nearly perfect matchings in regular simple hypergraphs , 1997 .

[28]  Aravind Srinivasan,et al.  Randomized Distributed Edge Coloring via an Extension of the Chernoff-Hoeffding Bounds , 1997, SIAM J. Comput..

[29]  Devdatt P. Dubhashi,et al.  Probabilistic Recurrence Relations Revisited , 1997, Theor. Comput. Sci..

[30]  Nabil Kahale.  Large Deviation Bounds for Markov Chains , 1997, Comb. Probab. Comput..

[31]  A. Dembo.  Information inequalities and concentration of measure , 1997 .

[32]  David A. Grable,et al.  A Large Deviation Inequality for Functions of Independent, Multi-Way Choices , 1998, Combinatorics, Probability and Computing.

[33]  R. Ravi,et al.  A polylogarithmic approximation algorithm for the group Steiner tree problem , 2000, SODA '98.

[34]  K. Marton.  Measure concentration for a class of random processes , 1998 .

[35]  Alessandro Panconesi,et al.  Fast distributed algorithms for Brooks-Vizing colourings , 2000, SODA '98.

[36]  Devdatt P. Dubhashi,et al.  Balls and bins: A study in negative dependence , 1996, Random Struct. Algorithms.

[37]  D. Gillman.  A Chernoff bound for random walks on expander graphs , 1998, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[38]  Alessandro Panconesi,et al.  Near-Optimal, Distributed Edge Colouring via the Nibble Method , 1996, Theor. Comput. Sci..

[39]  Y. Freund,et al.  Adaptive game playing using multiplicative weights , 1999 .

[40]  S. Bobkov,et al.  Exponential Integrability and Transportation Cost Related to Logarithmic Sobolev Inequalities , 1999 .

[41]  Dean P. Foster,et al.  Regret in the On-Line Decision Problem , 1999 .

[42]  Paul-Marie Samson,et al.  Concentration of measure inequalities for Markov chains and $\Phi$-mixing processes , 2000 .

[43]  Svante Janson,et al.  Random graphs , 2000, ZOR Methods Model. Oper. Res..

[44]  Russ Bubley,et al.  Randomized algorithms , 1995, CSUR.

[45]  M. Ledoux.  The concentration of measure phenomenon , 2001 .

[46]  B. Reed.  Graph Colouring and the Probabilistic Method , 2001 .

[47]  Colin McDiarmid.  Concentration For Independent Permutations , 2002, Comb. Probab. Comput..

[48]  Van H. Vu,et al.  Concentration of non‐Lipschitz functions and applications , 2002, Random Struct. Algorithms.

[49]  Svante Janson,et al.  The infamous upper tail , 2002, Random Struct. Algorithms.

[50]  Aravind Srinivasan,et al.  Approximation algorithms for the covering Steiner problem , 2002, Random Struct. Algorithms.

[51]  Jiri Matousek,et al.  Lectures on discrete geometry , 2002, Graduate texts in mathematics.

[52]  P. MassartLedoux,et al.  Concentration Inequalities Using the Entropy Method , 2002 .

[53]  Olle Häggström.  Finite Markov Chains and Algorithmic Applications , 2002 .

[54]  S. Boucheron,et al.  Concentration inequalities using the entropy method , 2003 .

[55]  Colin McDiarmid,et al.  Concentration for locally acting permutations , 2003, Discret. Math..

[56]  Gábor Lugosi,et al.  Concentration Inequalities , 2008, COLT.

[57]  Thomas P. Hayes.  Randomly coloring graphs of girth at least five , 2003, STOC '03.

[58]  Svante Janson,et al.  Large deviations for sums of partly dependent random variables , 2004, Random Struct. Algorithms.

[59]  Svante Janson,et al.  The Deletion Method For Upper Tail Estimates , 2004, Comb..

[60]  Jeffrey Considine,et al.  Geometric generalizations of the power of two choices , 2004, SPAA '04.

[61]  Jaikumar Radhakrishnan,et al.  Expansion properties of (secure) wireless networks , 2004, SPAA '04.

[62]  Roberto Di Pietro,et al.  Connectivity properties of secure wireless sensor networks , 2004, SASN '04.

[63]  Van H. Vu,et al.  Divide and conquer martingales and the number of triangles in a random graph , 2004, Random Struct. Algorithms.

[64]  Michael Mitzenmacher,et al.  Probability And Computing , 2005 .

[65]  Éva Tardos,et al.  Algorithm design , 2005 .

[66]  G. Lugosi.  Concentration-of-measure inequalities Lecture notes by Gábor Lugosi , 2005 .

[67]  Ali Esmaili,et al.  Probability and Random Processes , 2005, Technometrics.

[68]  S. Varadhan,et al.  Large deviations , 2019, Graduate Studies in Mathematics.

[69]  N. Berestycki,et al.  Concentration of measure , 2019, High-Dimensional Statistics.