Memory Flushing in Multijoin Query for Query Optimization

The enhancement of internet technology and advances in heterogeneous databases produced the requirement of database style querying over data from sources distributed around the world. When executing an online join query space is required in main memory to accommodate the data required to process the query. Memory of a specified amount is allotted to the query plan. When the main memory becomes full some data is required to move to disk to make room for new incoming tuples. Moving data from memory to disk is called Flushing. An optimal flushing strategy should be used to select the victim data so that the data which will least contribute in the future results would be flushed. All the previous algorithms are focused on flushing the memory when it becomes full but in this case some incoming data may be lost during the duration of flushing. In this paper we will propose a novel algorithm EarlyFlush to flush the memory before it becomes full to prevent the data loss and increase the throughput.

[1]  Jeffrey F. Naughton,et al.  Maximizing the Output Rate of Multi-Way Join Queries over Streaming Information Sources , 2003, VLDB.

[2]  Bernhard Seeger,et al.  On producing join results early , 2003, PODS '03.

[3]  Mohamed F. Mokbel,et al.  On Producing High and Early Result Throughput in Multijoin Query Plans , 2011, IEEE Transactions on Knowledge and Data Engineering.

[4]  Elke A. Rundensteiner,et al.  Run-time operator state spilling for memory intensive long-running queries , 2006, SIGMOD Conference.

[5]  Mohamed F. Mokbel,et al.  PermJoin: An Efficient Algorithm for Producing Early Results in Multi-join Query Plans , 2008, 2008 IEEE 24th International Conference on Data Engineering.

[6]  Yufei Tao,et al.  RPJ: producing fast join results on streams through rate-based optimization , 2005, SIGMOD '05.

[7]  Jennifer Widom,et al.  Memory-Limited Execution of Windowed Stream Joins , 2004, VLDB.

[8]  Goetz Graefe,et al.  Query evaluation techniques for large databases , 1993, CSUR.

[9]  Abraham Silberschatz,et al.  Database System Concepts , 1980 .

[10]  Walid G. Aref,et al.  Hash-merge join: a non-blocking join algorithm for producing fast and early join results , 2004, Proceedings. 20th International Conference on Data Engineering.

[11]  Jeffrey F. Naughton,et al.  Evaluating window joins over unbounded streams , 2003, Proceedings 19th International Conference on Data Engineering (Cat. No.03CH37405).

[12]  Yuguo Chen,et al.  On joining and caching stochastic streams , 2005, SIGMOD '05.