State access patterns in stream parallel computations

We introduce a set of state access patterns suitable for managing accesses to states in parallel computations operating on streams. The state access patterns are useful for modelling typical stream parallel applications. We present a classification of the patterns according to the extent and way in which the state can be structured and accessed. We define precisely the state access patterns and discuss possible implementation schemas, performances and possibilities to manage adaptivity (parallelism degree) in the patterns. We present experimental results relative to implementations built on top of the structured parallel programming framework FastFlow that demonstrate the feasibility and efficiency of the proposed access patterns.

[1]  Timothy G. Mattson,et al.  Patterns for parallel programming , 2004 .

[2]  Herbert Kuchen,et al.  On Implementing the Farm Skeleton , 2008, Parallel Process. Lett..

[3]  Bugra Gedik,et al.  Fundamentals of Stream Processing: Application Design, Systems, and Analytics , 2014 .

[4]  Tiziano De Matteis,et al.  Keep calm and react with foresight: strategies for low-latency and energy-efficient elastic data stream processing , 2016, PPoPP.

[5]  Henrique C. M. Andrade,et al.  Fundamentals of Stream Processing: Frontmatter , 2014 .

[6]  Raul Castro Fernandez,et al.  Integrating scale out and fault tolerance in stream processing using operator state management , 2013, SIGMOD '13.

[7]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[8]  Herbert Kuchen,et al.  Algorithmic skeletons for multi-core, multi-GPU systems and clusters , 2012, Int. J. High Perform. Comput. Netw..

[9]  Marco Danelutto,et al.  Deep Packet Inspection on Commodity Hardware using FastFlow , 2013, PARCO.

[10]  Mateo Valero,et al.  MultiLayer processing - an execution model for parallel stateful packet processing , 2008, ANCS '08.

[11]  James Demmel,et al.  the Parallel Computing Landscape , 2022 .

[12]  Marco Danelutto,et al.  Data stream processing via code annotations , 2016, The Journal of Supercomputing.

[13]  Raul Castro Fernandez,et al.  Scalable and Fault-tolerant Stateful Stream Processing , 2013, ICCSW.

[14]  Beng Chin Ooi,et al.  Parallelizing stateful operators in a distributed stream processing system: how, should you and how much? , 2012, DEBS.

[15]  Luis Miguel Sánchez,et al.  Discovering Pipeline Parallel Patterns in Sequential Legacy C++ Codes , 2016, PMAM@PPoPP.

[16]  De Giusti,et al.  Structured Parallel Programming: patterns for efficient computation , 2015 .

[17]  Bugra Gedik.  Partitioning functions for stateful data parallelism in stream processing , 2013, The VLDB Journal.

[18]  Christoph W. Kessler,et al.  SkePU: a multi-backend skeleton programming library for multi-GPU systems , 2010, HLPP '10.

[19]  Murray Cole,et al.  Bringing skeletons out of the closet: a pragmatic manifesto for skeletal parallel programming , 2004, Parallel Comput..

[20]  Marco Danelutto,et al.  Network Monitoring on Multicores with Algorithmic Skeletons , 2011, PARCO.

[21]  Marco Danelutto,et al.  Structured Parallel Programming with "core" FastFlow , 2013, CEFP.

[22]  Henrique C. M. Andrade,et al.  Fundamentals of Stream Processing by Henrique C. M. Andrade , 2014 .

[23]  Marco Danelutto,et al.  P3ARSEC: towards parallel patterns benchmarking , 2017, SAC.