A generic high throughput architecture for stream processing

Stream join is a fundamental and computationally expensive data mining operation for relating information from different data streams. This paper presents two FPGA-based architectures that accelerate stream join processing. The proposed hardware-based systems were implemented on a multi-FPGA hybrid system with high memory bandwidth. The experimental evaluation shows that our proposed systems can outperform a software-based solution that runs on a high-end, 48-core multiprocessor platform by at least one order of magnitude. In addition, the proposed solutions outperform any other previously proposed hardware-based or software-based solutions for stream join processing. Finally, our proposed hardware-based architectures can be used as generic templates to map stream processing algorithms on reconfigurable logic, taking into consideration real-world challenges and restrictions.

[1]  Jens Teubner,et al.  How soccer players would do stream joins , 2011, SIGMOD '11.

[2]  Anastasios Kementsietsidis,et al.  Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data , 2001, SIGMOD 2011.

[3]  Hideyuki Kawashima,et al.  An Efficient and Scalable Implementation of Sliding-Window Aggregate Operator on FPGA , 2013, 2013 First International Symposium on Computing and Networking.

[4]  Hans-Arno Jacobsen,et al.  SplitJoin: A Scalable, Low-latency Stream Join Architecture with Adjustable Ordering Precision , 2016, USENIX Annual Technical Conference.

[5]  Marina Papatriantafilou,et al.  Scalejoin: A deterministic, disjoint-parallel and skew-resilient stream join , 2015, 2015 IEEE International Conference on Big Data (Big Data).

[6]  Jun Yang,et al.  A Survey of Join Processing in Data Streams , 2007, Data Streams - Models and Algorithms.

[7]  Jens Teubner,et al.  Low-Latency Handshake Join , 2014, Proc. VLDB Endow..

[8]  Yisheng Dong,et al.  FPGA Acceleration Window Joins over Multiple Data Streams , 2005, J. Circuits Syst. Comput..

[9]  Philip S. Yu,et al.  CellJoin: a parallel stream join operator for the cell processor , 2009, The VLDB Journal.

[10]  Bharat Sukhwani,et al.  Accelerating Join Operation for Relational Databases with FPGAs , 2013, 2013 IEEE 21st Annual International Symposium on Field-Programmable Custom Computing Machines.

[11]  Hideyuki Kawashima,et al.  An Implementation of Handshake Join on FPGA , 2011, 2011 Second International Conference on Networking and Computing.

[12]  Dionisios N. Pnevmatikatos,et al.  An FPGA-based high-throughput stream join architecture , 2016, 2016 26th International Conference on Field Programmable Logic and Applications (FPL).

[13]  Jeffrey F. Naughton,et al.  Evaluating window joins over unbounded streams , 2003, Proceedings 19th International Conference on Data Engineering (Cat. No.03CH37405).