Towards highly parallel event processing through reconfigurable hardware

We present fpga-ToPSS (Toronto Publish/Subscribe System), an efficient event processing platform to support high-frequency and low-latency event matching. fpga-ToPSS is built over reconfigurable hardware---FPGAs---to achieve line-rate processing by exploring various degrees of parallelism. Furthermore, each of our proposed FPGA-based designs is geared towards a unique application requirement, such as flexibility, adaptability, scalability, or pure performance, such that each solution is specifically optimized to attain a high level of parallelism. Therefore, each solution is formulated as a design trade-off between the degree of parallelism versus the desired application requirement. Moreover, our event processing engine supports Boolean expression matching with an expressive predicate language applicable to a wide range of applications including real-time data analysis, algorithmic trading, targeted advertisement, and (complex) event processing.

[1]  Hans-Arno Jacobsen,et al.  GPX-matcher: a generic boolean predicate-based XPath expression matcher , 2011, EDBT/ICDT '11.

[2]  Hans-Arno Jacobsen,et al.  Predicate matching and subscription matching in Publish/Subscribe systems , 2002, Proceedings 22nd International Conference on Distributed Computing Systems Workshops.

[3]  Theodore Johnson,et al.  Gigascope: a stream database for network applications , 2003, SIGMOD '03.

[4]  Yanlei Diao,et al.  High-performance complex event processing over streams , 2006, SIGMOD Conference.

[5]  Glen Gibb,et al.  NetFPGA--An Open Platform for Gigabit-Rate Network Switching and Routing , 2007, 2007 IEEE International Conference on Microelectronic Systems Education (MSE'07).

[6]  Hans-Arno Jacobsen,et al.  Efficient event processing through reconfigurable hardware for algorithmic trading , 2010, Proc. VLDB Endow..

[7]  Gustavo Alonso,et al.  Streams on Wires - A Query Compiler for FPGAs , 2009, Proc. VLDB Endow..

[8]  Michael Stonebraker,et al.  OLTP through the looking glass, and what we found there , 2008, SIGMOD Conference.

[9]  Babak Falsafi,et al.  Shore-MT: a scalable storage manager for the multicore era , 2009, EDBT '09.

[10]  Hans-Arno Jacobsen,et al.  Towards vulnerability-based intrusion detection with event processing , 2011, DEBS '11.

[11]  Marcos K. Aguilera,et al.  Matching events in a content-based subscription system , 1999, PODC '99.

[12]  Johannes Gehrke,et al.  Cayuga: a high-performance event processing engine , 2007, SIGMOD '07.

[13]  Sergei Vassilvitskii,et al.  Efficiently evaluating complex boolean expressions , 2010, SIGMOD Conference.

[14]  Petko Bakalov,et al.  Boosting XML Filtering with a Scalable FPGA-based Architecture , 2009, CIDR 2009.

[15]  Yanlei Diao,et al.  YFilter: efficient and scalable filtering of XML documents , 2002, Proceedings 18th International Conference on Data Engineering.

[16]  Gustavo Alonso,et al.  Complex event detection at wire speed with FPGAs , 2010, Proc. VLDB Endow..

[17]  J. Gregory Steffan,et al.  Improving Pipelined Soft Processors with Multithreading , 2007, 2007 International Conference on Field Programmable Logic and Applications.

[18]  Dennis Shasha,et al.  Filtering algorithms and implementation for very fast publish/subscribe systems , 2001, SIGMOD '01.

[19]  Hans-Arno Jacobsen,et al.  A Unified Approach to Routing, Covering and Merging in Publish/Subscribe Systems Based on Modified Binary Decision Diagrams , 2005, 25th IEEE International Conference on Distributed Computing Systems (ICDCS'05).

[20]  Divesh Srivastava,et al.  Enabling real time data analysis , 2010, Proc. VLDB Endow..

[21]  Ippokratis Pandis,et al.  Data-oriented transaction execution , 2010, Proc. VLDB Endow..

[22]  Wayne Luk,et al.  FPGA Accelerated Low-Latency Market Data Feed Processing , 2009, 2009 17th IEEE Symposium on High Performance Interconnects.

[23]  Russell Tessier,et al.  FPGA Architecture: Survey and Challenges , 2008, Found. Trends Electron. Des. Autom..

[24]  Hans-Arno Jacobsen,et al.  Processing proximity relations in road networks , 2010, SIGMOD Conference.

[25]  Hans-Arno Jacobsen,et al.  BE-tree: an index structure to efficiently match boolean expressions over high-dimensional discrete space , 2011, SIGMOD '11.