Optimal Lower Bounds for Universal Relation, and for Samplers and Finding Duplicates in Streams

In the communication problem UR (universal relation), Alice and Bob respectively receive x, y ∊{0,1\}^n with the promise that x≠ y. The last player to receive a message must output an index i such that x_i≠ y_i. We prove that the randomized one-way communication complexity of this problem in the public coin model is exactly \Theta(\min\{n,\log(1/δ)\log^2(\frac n{\log(1/δ)})\}) for failure probability δ. Our lower bound holds even if promised \mathop{support}(y)⊄ \mathop{support}(x). As a corollary, we obtain optimal lower bounds for ℓ_p-sampling in strict turnstile streams for 0\le p streams for 0 ≤ p

[1]  Sudipto Guha,et al.  Spectral Sparsification in Dynamic Graph Streams , 2013, APPROX-RANDOM.

[2]  Funda Ergün,et al.  Periodicity in Streams , 2010, APPROX-RANDOM.

[3]  Alexandr Andoni,et al.  Streaming Algorithms via Precision Sampling , 2010, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[4]  Peter Robinson,et al.  Fast Distributed Algorithms for Connectivity and MST in Large Graphs , 2015, SPAA.

[5]  Mohammad Taghi Hajiaghayi,et al.  Parameterized Streaming: Maximal Matching and Vertex Cover , 2015, SODA.

[6]  Yang Li,et al.  On Estimating Maximum Matching Size in Graph Streams , 2017, SODA.

[7]  Martin Farach-Colton,et al.  Tight Approximations of Degeneracy in Large Graphs , 2016, LATIN.

[8]  David P. Woodruff,et al.  Optimal Bounds for Johnson-Lindenstrauss Transforms and Streaming Problems with Subconstant Error , 2011, TALG.

[9]  Bruce M. Kapron,et al.  Dynamic graph connectivity in polylogarithmic worst case time , 2013, SODA.

[10]  Sudipto Guha,et al.  Graph sketches: sparsification, spanners, and subgraphs , 2012, PODS.

[11]  David P. Woodruff,et al.  An optimal algorithm for the distinct elements problem , 2010, PODS '10.

[12]  Jun Tarui,et al.  Finding a Duplicate and a Missing Item in a Stream , 2007, TAMC.

[13]  Yang Li,et al.  Maximum Matchings in Dynamic Graph Streams and the Simultaneous Communication Model , 2016, SODA.

[14]  Bruce M. Kapron,et al.  Dynamic graph connectivity with improved worst case update time and sublinear space , 2015, ArXiv.

[15]  Avi Wigderson,et al.  Monotone circuits for connectivity require super-logarithmic depth , 1990, STOC '88.

[16]  Russell Impagliazzo,et al.  Communication complexity towards lower bounds on circuit depth , 2001, computational complexity.

[17]  Sofya Vorotnikova,et al.  Kernelization via Sampling with Applications to Finding Matchings and Related Problems in Dynamic Graph Streams , 2016, SODA.

[18]  Peter Bro Miltersen,et al.  On data structures and asymmetric communication complexity , 1994, STOC '95.

[19]  Charalampos E. Tsourakakis,et al.  Space- and Time-Efficient Algorithm for Maintaining Dense Subgraphs on One-Pass Dynamic Streams , 2015, STOC.

[20]  Yin Tat Lee,et al.  Single Pass Spectral Sparsification in Dynamic Streams , 2014, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[21]  Avi Wigderson,et al.  Composition of the Universal Relation , 1990, Advances In Computational Complexity Theory.

[22]  Hossein Jowhari,et al.  Tight bounds for Lp samplers, finding duplicates in streams, and related problems , 2010, PODS.

[23]  Piotr Indyk,et al.  Sampling in dynamic data streams and applications , 2005, Int. J. Comput. Geom. Appl..

[24]  Graham Cormode,et al.  Summarizing and Mining Inverse Distributions on Data Streams via Dynamic Inverse Sampling , 2005, VLDB.

[25]  Uri Zwick,et al.  The communication complexity of the universal relation , 1997, Proceedings of Computational Complexity. Twelfth Annual IEEE Conference.

[26]  Jakub W. Pachocki,et al.  Optimal lower bounds for universal relation, samplers, and finding duplicates , 2017, ArXiv.

[27]  Ravi Kumar,et al.  An improved data stream algorithm for frequency moments , 2004, SODA '04.

[28]  Sudipto Guha,et al.  Analyzing graph structure via linear measurements , 2012, SODA.

[29]  David P. Woodruff,et al.  1-pass relative-error Lp-sampling with applications , 2010, SODA '10.

[30]  Sofya Vorotnikova,et al.  Densest Subgraph in Dynamic Graph Streams , 2015, MFCS.

[31]  Andrew McGregor,et al.  Graph stream algorithms: a survey , 2014, SGMD.

[32]  Chris Schwiegelshohn,et al.  Sublinear Estimation of Weighted Matchings in Dynamic Data Streams , 2015, ESA.

[33]  Graham Cormode,et al.  A unifying framework for ℓ0-sampling algorithms , 2013, Distributed and Parallel Databases.

[34]  Or Meir,et al.  Toward better formula lower bounds: an information complexity approach to the KRW composition conjecture , 2014, STOC.

[35]  Jaikumar Radhakrishnan,et al.  Finding duplicates in a data stream , 2009, SODA.

[36]  Zhengyu Wang.  An Improved Randomized Data Structure for Dynamic Graph Connectivity , 2015, ArXiv.

[37]  Ran Raz,et al.  Super-logarithmic depth lower bounds via direct sum in communication complexity , 1991, [1991] Proceedings of the Sixth Annual Structure in Complexity Theory Conference.

[38]  Sudipto Guha,et al.  Vertex and Hyperedge Connectivity in Dynamic Graph Streams , 2015, PODS.

[39]  Or Meir,et al.  Toward the KRW Composition Conjecture: Cubic Formula Lower Bounds via Communication Complexity , 2016, Electron. Colloquium Comput. Complex..

[40]  Christian Konrad,et al.  Maximum Matching in Turnstile Streams , 2015, ESA.

[41]  S. Muthukrishnan,et al.  Data streams: algorithms and applications , 2005, SODA '03.

[42]  Sriram V. Pemmaraju,et al.  Toward Optimal Bounds in the Congested Clique: Graph Connectivity and MST , 2015, PODC.

[43]  David P. Woodruff,et al.  Brief Announcement: Applications of Uniform Sampling: Densest Subgraph and Beyond , 2015, SPAA.