FreshJoin: An Efficient and Adaptive Algorithm for Set Containment Join

This paper revisits set containment join (SCJ), which has many fundamental applications in commercial and scientific fields. To improve the performance further, this paper proposes a new adaptive parameter-free in-memory algorithm for SCJ, named as \(\mathsf {FreshJoin}\). It accomplishes this by exploiting two flat indices, which record three kinds of signatures (i.e., the two least frequent elements and a hash signature). Experiments on 16 real-life datasets show that \(\mathsf {FreshJoin}\) usually reduces more than 50% of space costs while remains as competitive as the state-of-the-art algorithms in running time.

[1]  Vipin Kumar,et al.  Multi-view ensemble learning: an optimal feature set partitioning for high-dimensional data classification , 2015, Knowledge and Information Systems.

[2]  Stefanie Rinderle-Ma,et al.  Dealing with change in process choreographies: Design and implementation of propagation algorithms☆ , 2015, Inf. Syst..

[3]  Nikos Mamoulis,et al.  Set containment join revisited , 2015, Knowledge and Information Systems.

[4]  Hector Garcia-Molina,et al.  Divide-and-Conquer Algorithm for Computing Set Containment Joins , 2002, EDBT.

[5]  Renée J. Miller,et al.  LSH Ensemble: Internet-Scale Domain Search , 2016, Proc. VLDB Endow..

[6]  Jan Hidders,et al.  Efficient and scalable trie-based algorithms for computing set containment relations , 2015, 2015 IEEE 31st International Conference on Data Engineering.

[7]  Nikos Mamoulis,et al.  Efficient processing of joins on set-valued attributes , 2003, SIGMOD '03.

[8]  Hector Garcia-Molina,et al.  Adaptive algorithms for set containment joins , 2003, TODS.

[9]  Xuemin Lin,et al.  TT-Join: Efficient Set Containment Join , 2017, 2017 IEEE 33rd International Conference on Data Engineering (ICDE).

[10]  Nikolaus Augsten,et al.  An Empirical Evaluation of Set Similarity Join Techniques , 2016, Proc. VLDB Endow..

[11]  Jeffrey F. Naughton,et al.  Set Containment Joins: The Good, The Bad and The Ugly , 2000, VLDB.

[12]  Vikram Pudi,et al.  Using Prefix-Trees for Efficiently Computing Set Joins , 2005, DASFAA.

[13]  Sven Helmer,et al.  PIEJoin: Towards Parallel Set Containment Joins , 2016, SSDBM.

[14]  Sven Helmer,et al.  Evaluation of Main Memory Join Algorithms for Joins with Set Comparison Join Predicates , 1996, VLDB.