Sampling in Space Restricted Settings

Space efficient algorithms play an important role in dealing with large amount of data. In such settings, one would like to analyze the large data using small amount of “working space”. One of the key steps in many algorithms for analyzing large data is to maintain a (or a small number) random sample from the data points. In this paper, we consider two space restricted settings—(i) the streaming model, where data arrives over time and one can use only a small amount of storage, and (ii) the query model, where we can structure the data in low space and answer sampling queries. In this paper, we prove the following results in the above two settings:In the streaming setting, we would like to maintain a random sample from the elements seen so far. We prove that one can maintain a random sample using $$O(\log n)$$O(logn) random bits and $$O(\log n)$$O(logn) bits of space, where n is the number of elements seen so far. We can extend this to the case when elements have weights as well.In the query model, there are n elements with weights $$w_1, \ldots , w_n$$w1,…,wn (which are w-bit integers) and one would like to sample a random element with probability proportional to its weight. Bringmann and Larsen (STOC 2013) showed how to sample such an element using $$nw +1 $$nw+1 bits of space (whereas, the information theoretic lower bound is nw). We consider the approximate sampling problem, where we are given an error parameter $$\varepsilon $$ε, and the sampling probability of an element can be off by an $$\varepsilon $$ε factor. We give matching upper and lower bounds for this problem.

[1]  David Thomas,et al.  The Art in Computer Programming , 2001 .

[2]  Amit Kumar,et al.  A Simple D 2-Sampling Based PTAS for k-Means and other Clustering Problems , 2012, COCOON.

[3]  A. J. Walker New fast method for generating discrete random numbers with arbitrary frequency distributions , 1974 .

[4]  Jeffrey Scott Vitter,et al.  Faster methods for random sampling , 1984, CACM.

[5]  Paul G. Spirakis,et al.  Weighted random sampling with a reservoir , 2006, Inf. Process. Lett..

[6]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[7]  Kim-Hung Li,et al.  Reservoir-sampling algorithms of time complexity O(n(1 + log(N/n))) , 1994, TOMS.

[8]  Rajeev Motwani,et al.  Sampling from a moving window over streaming data , 2002, SODA '02.

[9]  Donald E. Knuth,et al.  The art of computer programming, volume 3: (2nd ed.) sorting and searching , 1998 .

[10]  Kasper Green Larsen,et al.  Succinct sampling from discrete distributions , 2013, STOC '13.

[11]  Nagiza F. Samatova,et al.  Sampling streaming data with replacement , 2007, Comput. Stat. Data Anal..

[12]  Jeffrey Scott Vitter,et al.  Random sampling with a reservoir , 1985, TOMS.

[13]  Konstantinos Panagiotou,et al.  Efficient Sampling Methods for Discrete Distributions , 2012, ICALP.

[14]  Amit Kumar,et al.  A Simple D2-Sampling Based PTAS for k-Means and Other Clustering Problems , 2012, Algorithmica.

[15]  R. Kronmal,et al.  On the Alias Method for Generating Random Variables From a Discrete Distribution , 1979 .