Random sampling is an appealing approach to build synopses of large data streams because random samples can be used for a broad spectrum of analytical tasks. Users are often interested in analyzing only the most recent fraction of the data stream in order to avoid outdated results. In this paper, we focus on sampling schemes that sample from a sliding window over a recent time interval; such windows are a popular and highly comprehensible method to model recency. In this setting, the main challenge is to guarantee an upper bound on the space consumption of the sample while using the allotted space efficiently at the same time. The difficulty arises from the fact that the number of items in the window is unknown in advance and may vary significantly over time, so that the sampling fraction has to be adjusted dynamically. We consider uniform sampling schemes, which produce each sample of the same size with equal probability, and stratified sampling schemes, in which the window is divided into smaller strata and a uniform sample is maintained per stratum. For uniform sampling, we prove that it is impossible to guarantee a minimum sample size in bounded space. We then introduce a novel sampling scheme called bounded priority sampling (BPS), which requires only bounded space. We derive a lower bound on the expected sample size and show that BPS quickly adapts to changing data rates. For stratified sampling, we propose a merge-based stratification scheme (MBS), which maintains strata of approximately equal size. Compared to naive stratification, MBS has the advantage that the sample is evenly distributed across the window, so that no part of the window is over- or underrepresented. We conclude the paper with a feasibility study of our algorithms on large real-world datasets.
[1]
Torsten Suel,et al.
Optimal Histograms with Quality Guarantees
,
1998,
VLDB.
[2]
Yossi Matias,et al.
Fast incremental maintenance of approximate histograms
,
1997,
TODS.
[3]
Peter J. Haas,et al.
Techniques for Warehousing of Sample Data
,
2006,
22nd International Conference on Data Engineering (ICDE'06).
[4]
Peter J. Haas,et al.
Maintaining bounded-size sample synopses of evolving datasets
,
2008,
The VLDB Journal.
[5]
Carl-Erik Särndal,et al.
Model Assisted Survey Sampling
,
1997
.
[6]
Rajeev Motwani,et al.
Towards estimation error guarantees for distinct values
,
2000,
PODS.
[7]
Peter J. Haas,et al.
On synopses for distinct-value estimation under multiset operations
,
2007,
SIGMOD '07.
[8]
Jeffrey Scott Vitter,et al.
Faster methods for random sampling
,
1984,
CACM.
[9]
Piotr Indyk,et al.
Maintaining Stream Statistics over Sliding Windows
,
2002,
SIAM J. Comput..
[10]
Rajeev Motwani,et al.
Sampling from a moving window over streaming data
,
2002,
SODA '02.
[11]
Cecilia R. Aragon,et al.
Randomized search trees
,
2005,
Algorithmica.
[12]
Srinivasan Seshan,et al.
Synopsis diffusion for robust aggregation in sensor networks
,
2004,
SenSys '04.
[13]
Charu C. Aggarwal,et al.
On biased reservoir sampling in the presence of stream evolution
,
2006,
VLDB.
[14]
Jeffrey Scott Vitter,et al.
Random sampling with a reservoir
,
1985,
TOMS.