An optimal lower bound on the communication complexity of gap-hamming-distance

We prove an optimal Ω(n) lower bound on the randomized communication complexity of the much-studied Gap-Hamming-Distance problem. As a consequence, we obtain essentially optimal multi-pass space lower bounds in the data stream model for a number of fundamental problems, including the estimation of frequency moments. The Gap-Hamming-Distance problem is a communication problem, wherein Alice and Bob receive n-bit strings x and y, respectively. They are promised that the Hamming distance between x and y is either at least n/2+√n or at most n/2-√n, and their goal is to decide which of these is the case. Since the formal presentation of the problem by Indyk and Woodruff (FOCS, 2003), it had been conjectured that the naive protocol, which uses n bits of communication, is asymptotically optimal. The conjecture was shown to be true in several special cases, e.g., when the communication is deterministic, or when the number of rounds of communication is limited. The proof of our aforementioned result, which settles this conjecture fully, is based on a new geometric statement regarding correlations in Gaussian space, related to a result of C. Borell (1985). To prove this geometric statement, we show that random projections of not-too-small sets in Gaussian space are close to a mixture of translated normal variables.

[1]  Graham Cormode,et al.  A near-optimal algorithm for estimating the entropy of a stream , 2010, TALG.

[2]  Peter Bro Miltersen,et al.  On data structures and asymmetric communication complexity , 1994, STOC '95.

[3]  Joshua Brody,et al.  Property Testing Lower Bounds via Communication Complexity , 2011, 2011 IEEE 26th Annual Conference on Computational Complexity.

[4]  Alexander A. Razborov,et al.  On the Distributional Complexity of Disjointness , 1992, Theor. Comput. Sci..

[5]  Franck Barthe.  An Isoperimetric Result for the Gaussian Measure and Unconditional Sets , 2001 .

[6]  Hartmut Klauck,et al.  The Partition Bound for Classical Communication Complexity and Query Complexity , 2009, 2010 IEEE 25th Annual Conference on Computational Complexity.

[7]  David P. Woodruff,et al.  Tight bounds for distributed functional monitoring , 2011, STOC '12.

[8]  Noga Alon,et al.  The Space Complexity of Approximating the Frequency Moments , 1999 .

[9]  Graham Cormode,et al.  A near-optimal algorithm for computing the entropy of a stream , 2007, SODA '07.

[10]  Avi Wigderson,et al.  A Strong Direct Product Theorem for Corruption and the Multiparty Communication Complexity of Disjointness , 2006, computational complexity.

[11]  David P. Woodruff.  Optimal space lower bounds for all frequency moments , 2004, SODA '04.

[12]  Felix Wu,et al.  The quantum query complexity of approximating the median and related statistics , 1998, STOC '99.

[13]  A. Razborov.  Quantum communication complexity of symmetric predicates , 2002, quant-ph/0204025.

[14]  D. Freedman,et al.  Asymptotics of Graphical Projection Pursuit , 1984 .

[15]  Alexander A. Sherstov.  The Pattern Matrix Method , 2009, SIAM J. Comput..

[16]  Ravi Kumar,et al.  The One-Way Communication Complexity of Hamming Distance , 2008, Theory Comput..

[17]  David P. Woodruff,et al.  Tight lower bounds for the distinct elements problem , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[18]  B. Klartag.  A central limit theorem for convex sets , 2006, math/0605014.

[19]  David P. Woodruff.  Efficient and private distance approximation in the communication and streaming models , 2007 .

[20]  C. Borell.  Geometric bounds on the Ornstein-Uhlenbeck velocity process , 1985 .

[21]  Alexander A. Sherstov.  The Communication Complexity of Gap Hamming Distance , 2012, Theory Comput..

[22]  David P. Woodru.  Ecient and Private Distance Approximation in the Communication and Streaming Models , 2007 .

[23]  Tsuyoshi Ito,et al.  A concentration inequality for the overlap of a vector on a large set , with application to communication complexity , 2010 .

[24]  Joshua Brody,et al.  Better Gap-Hamming Lower Bounds via Better Round Elimination , 2010, APPROX-RANDOM.

[25]  A. Razborov.  Communication Complexity , 2011 .

[26]  Toniann Pitassi,et al.  The Limits of Two-Party Differential Privacy , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[27]  Chrisil Arackaparambil,et al.  Functional Monitoring without Monotonicity , 2009, ICALP.

[28]  David P. Woodruff,et al.  On the exact space complexity of sketching and streaming small norms , 2010, SODA '10.

[29]  K. Ball.  An elementary introduction to modern convex geometry, in flavors of geometry , 1997 .

[30]  David P. Woodruff,et al.  An optimal algorithm for the distinct elements problem , 2010, PODS '10.

[31]  R. Gray.  Entropy and Information Theory , 1990, Springer New York.

[32]  Alexander A. Sherstov.  The pattern matrix method for lower bounds on quantum communication , 2008, STOC '08.

[33]  David P. Woodruff.  The average-case complexity of counting distinct elements , 2009, ICDT '09.

[34]  K. Ball.  An Elementary Introduction to Modern Convex Geometry , 1997 .

[35]  Thomas Vidick,et al.  A concentration inequality for the overlap of a vector on a large set, with application to the communication complexity of the Gap-Hamming-Distance problem , 2011, Chic. J. Theor. Comput. Sci..

[36]  Avi Wigderson,et al.  Quantum vs. classical communication and computation , 1998, STOC '98.

[37]  Hartmut Klauck.  A strong direct product theorem for disjointness , 2010, STOC '10.

[38]  Nathan Linial,et al.  Lower bounds in communication complexity based on factorization norms , 2007, STOC '07.

[39]  Avi Wigderson,et al.  Monotone circuits for connectivity require super-logarithmic depth , 1990, STOC '88.

[40]  Luca Trevisan,et al.  Counting Distinct Elements in a Data Stream , 2002, RANDOM.

[41]  Ran Raz,et al.  Exponential separation of quantum and classical communication complexity , 1999, STOC '99.

[42]  Fabian Kuhn,et al.  The Complexity of Data Aggregation in Directed Networks , 2011, DISC.

[43]  Mihai Patrascu.  (Data) STRUCTURES , 2008, FOCS.

[44]  Sergey G. Bobkov,et al.  On concentration of distributions of random weighted sums , 2003 .

[45]  Hartmut Klauck,et al.  Rectangle size bounds and threshold covers in communication complexity , 2002, 18th IEEE Annual Conference on Computational Complexity, 2003. Proceedings..

[46]  Amit Chakrabarti,et al.  An Optimal Lower Bound on the Communication Complexity of Gap-Hamming-Distance , 2012, SIAM J. Comput..

[47]  Joshua Brody,et al.  A Multi-Round Communication Lower Bound for Gap Hamming and Some Consequences , 2009, 2009 24th Annual IEEE Conference on Computational Complexity.