The Complexity of Querying External Memory and Streaming Data

We review a recently introduced computation model for streaming and external memory data. An important feature of this model is that it distinguishes between sequentially reading (streaming) data from external memory (through main memory) and randomly accessing external memory data at specific memory locations; it is well-known that the latter is much more expensive in practice. We explain how a number of lower bound results are obtained in this model and how they can be applied for proving lower bounds for XML query processing.

[1]  Jennifer Widom,et al.  Characterizing memory requirements for queries over continuous data streams , 2002, PODS '02.

[2]  Mayur Datar,et al.  On the streaming model augmented with a sorting primitive , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[3]  Georg Gottlob,et al.  The complexity of XPath query evaluation , 2003, PODS.

[4]  Nicole Schweikardt,et al.  Tight lower bounds for query processing on streaming and external memory data , 2005, Theor. Comput. Sci..

[5]  Marcus Fontoura,et al.  On the memory requirements of XPath evaluation over XML streams , 2004, PODS.

[6]  Marcus Fontoura,et al.  Buffering in query evaluation over XML streams , 2005, PODS '05.

[7]  S. Muthukrishnan,et al.  Data streams: algorithms and applications , 2005, SODA '03.

[8]  Jeffrey Scott Vitter.  External memory algorithms , 1998, PODS '98.

[9]  Jop F. Sibeyn,et al.  Algorithms for Memory Hierarchies: Advanced Lectures , 2003 .

[10]  Helmut Seidl,et al.  Locating Matches of Tree Patterns in Forests , 1998, FSTTCS.

[11]  Georg Gottlob,et al.  Efficient Algorithms for Processing XPath Queries , 2002, VLDB.

[12]  Raghu Ramakrishnan,et al.  Database Management Systems , 1976 .

[13]  J. Ian Munro,et al.  Selection and sorting with limited storage , 1978, 19th Annual Symposium on Foundations of Computer Science (sfcs 1978).

[14]  Prabhakar Raghavan,et al.  Computing on data streams , 1999, External Memory Algorithms.

[15]  Alexander A. Razborov,et al.  Applications of matrix methods to the theory of lower bounds in computational complexity , 1990, Comb..

[16]  Luc Segoufin,et al.  Typing and querying XML documents: some complexity bounds , 2003, PODS.

[17]  Jennifer Widom,et al.  Models and issues in data stream systems , 2002, PODS.

[18]  Zvi Galil,et al.  Lower bounds on communication complexity , 1984, STOC '84.

[19]  Jeffrey Scott Vitter,et al.  External memory algorithms and data structures: dealing with massive data , 2001, CSUR.

[20]  Nicole Schweikardt,et al.  Tight lower bounds for query processing on streaming and external memory data , 2007, Theor. Comput. Sci..

[21]  Goetz Graefe,et al.  Query evaluation techniques for large databases , 1993, CSUR.

[22]  Andrew Chi-Chih Yao,et al.  Some complexity questions related to distributive computing(Preliminary Report) , 1979, STOC.

[23]  Ulrich Meyer,et al.  Algorithms for Memory Hierarchies , 2003, Lecture Notes in Computer Science.

[24]  Nicole Schweikardt,et al.  Lower bounds for sorting with few random accesses to external memory , 2005, PODS.

[25]  Victor Vianu,et al.  Validating streaming XML documents , 2002, PODS.

[26]  Noga Alon,et al.  The Space Complexity of Approximating the Frequency Moments , 1999 .