New Characterizations in Turnstile Streams with Applications

Recently, [Li, Nguyen, Woodruff, STOC'2014] showed any 1-pass constant probability streaming algorithm for computing a relation f on a vector x ∈ {−m, − (m − 1), ..., m}n presented in the turnstile data stream model can be implemented by maintaining a linear sketch A · × mod q, where A is an r × n integer matrix and q = (q1, ..., qr) is a vector of positive integers. The space complexity of maintaining A · × mod q, not including the random bits used for sampling A and q, matches the space of the optimal algorithm. We give multiple strengthenings of this reduction, together with new applications. In particular, we show how to remove the following shortcomings of their reduction: 1. The Box Constraint. Their reduction applies only to algorithms that must be correct even if ∥;x∥;∞ = maxi∈[n] |xi| is allowed to be much larger than m at intermediate points in the stream, provided that x ∈ {−m, −(m − 1), ..., m}n at the end of the stream. We give a condition under which the optimal algorithm is a linear sketch even if it works only when promised that x ∈ {−m, −(m − 1), ..., m}n at all points in the stream. Using this, we show the first super-constant Ω(log m) bits lower bound for the problem of maintaining a counter up to an additive em error in a turnstile stream, where e is any constant in (0, ½). Previous lower bounds are based on communication complexity and are only for relative error approximation; interestingly, we do not know how to prove our result using communication complexity. More generally, we show the first super-constant Ω(log m) lower bound for additive approximation of ep-norms; this bound is tight for 1 ≤ p ≤ 2. 2. Negative Coordinates. Their reduction allows xi to be negative while processing the stream. We show an equivalence between 1-pass algorithms and linear sketches A · x mod q in dynamic graph streams, or more generally, the strict turnstile model, in which for all i ∈ [n], xi ≥ 0 at all points in the stream. Combined with [Assadi, Khanna, Li, Yaroslavtsev, SODA'2016], this resolves the 1-pass space complexity of approximating the maximum matching in a dynamic graph stream, answering a question in that work. 3. 1-Pass Restriction. Their reduction only applies to 1-pass data stream algorithms in the turnstile model, while there exist algorithms for heavy hitters and for low rank approximation which provably do better with multiple passes. We extend the reduction to algorithms which make any number of passes, showing the optimal algorithm is to choose a new linear sketch at the beginning of each pass, based on the output of previous passes.