Should PARSEC Benchmarks be More Parametric? A Case Study with Dedup

Parallel applications of the same domain can present similar patterns of behavior and characteristics. Characterizing common application behaviors can help for understanding performance aspects in the real-world scenario. One way to better understand and evaluate applications' characteristics is by using customizable/parametric benchmarks that enable users to represent important characteristics at run-time. We observed that parameterization techniques should be better exploited in the available benchmarks, especially on stream processing domain. For instance, although widely used, the stream processing benchmarks available in PARSEC do not support the simulation and evaluation of relevant and modern characteristics. Therefore, our goal is to identify the stream parallelism characteristics present in PARSEC. We also implemented a ready to use parameterization support and evaluated the application behaviors considering relevant performance metrics for stream parallelism (service time, throughput, latency). We choose Dedup to be our case study. The experimental results have shown performance improvements in our parameterization support for Dedup. Moreover, this support increased the customization space for benchmark users, which is simple to use. In the future, our solution can be potentially explored on different parallel architectures and parallel programming frameworks.

[1]  Kunle Olukotun,et al.  Eigenbench: A simple exploration tool for orthogonal TM characteristics , 2010, IEEE International Symposium on Workload Characterization (IISWC'10).

[2]  Robert Grimm,et al.  A catalog of stream processing optimizations , 2014, ACM Comput. Surv..

[3]  Marco Danelutto,et al.  Autonomic and Latency-Aware Degree of Parallelism Management in SPar , 2018, Euro-Par Workshops.

[4]  William Thies,et al.  StreamIt: A Language for Streaming Applications , 2002, CC.

[5]  Jóakim von Kistowski,et al.  How to Build a Benchmark , 2015, ICPE.

[6]  Rafael Asenjo,et al.  Analytical Modeling of Pipeline Parallelism , 2009, 2009 18th International Conference on Parallel Architectures and Compilation Techniques.

[7]  Marco Danelutto,et al.  High-Level and Productive Stream Parallelism for Dedup, Ferret, and Bzip2 , 2018, International Journal of Parallel Programming.

[8]  David Mazières,et al.  A low-bandwidth network file system , 2001, SOSP.

[9]  Marco Danelutto,et al.  Bringing Parallel Patterns Out of the Corner , 2017, ACM Trans. Archit. Code Optim..

[10]  Christian Bienia,et al.  Benchmarking modern multiprocessors , 2011 .

[11]  William Thies,et al.  An empirical characterization of stream programs and its implications for language and compiler design , 2010, 2010 19th International Conference on Parallel Architectures and Compilation Techniques (PACT).

[12]  Eduard Ayguadé,et al.  PARSECSs: Evaluating the Impact of Task Parallelism in the PARSEC Benchmark Suite , 2016, ACM Trans. Archit. Code Optim..

[13]  Kai Li,et al.  PARSEC3.0: A Multicore Benchmark Suite with Network Stacks and SPLASH-2X , 2017, CARN.