Performance Characterization of Spark Workloads on Shared NUMA Systems

As the adoption of Big Data technologies becomes the norm in an increasing number of scenarios, there is also a growing need to optimize them for modern processors. Spark has gained momentum over the last few years among companies looking for high performance solutions that can scale out across different cluster sizes. At the same time, modern processors can be connected to large amounts of physical memory, in the range of up to few terabytes. This opens an enormous range of opportunities for runtimes and applications that aim to improve their performance by leveraging low latencies and high bandwidth provided by RAM. The result is that there are several examples today of applications that have started pushing the in-memory computing paradigm to accelerate tasks. To deliver such a large physical memory capacity, hardware vendors have leveraged Non-Uniform Memory Architectures (NUMA). This paper explores how Spark-based workloads are impacted by the effects of NUMA-placement decisions, how different Spark configurations result in changes in delivered performance, how the characteristics of the applications can be used to predict workload collocation conflicts, and how to improve performance by collocating workloads in scale-up nodes. We explore several workloads run on top of the IBM Power8 processor, and provide manual strategies that can leverage performance improvements up to 40% on Spark workloads when using smart processor-pinning and workload collocation strategies.

[1]  Michael Gschwind,et al.  IBM POWER8 processor core microarchitecture , 2015, IBM J. Res. Dev..

[2]  Vladimir Vlassov,et al.  Architectural Impact on Performance of In-memory Data Analytics: Apache Spark Case Study , 2016, ArXiv.

[3]  Norman May,et al.  Adaptive NUMA-aware data placement and task scheduling for analytical workloads in main-memory column-stores , 2016, Proc. VLDB Endow..

[4]  Haibo Chen,et al.  NUMA-aware graph-structured analytics , 2015, PPoPP.

[5]  Alexandra Fedorova,et al.  A case for NUMA-aware contention management on multicore systems , 2010, 2010 19th International Conference on Parallel Architectures and Compilation Techniques (PACT).

[6]  Jaehyuk Huh,et al.  Dynamic Virtual Machine Scheduling in Clouds for Architectural Shared Resources , 2012, HotCloud.

[7]  Timothée Ewart,et al.  Performance evaluation of the IBM POWER8 architecture to support computational neuroscientific application using morphologically detailed neurons , 2015, PMBS '15.

[8]  Karine Heydemann,et al.  Scalable task parallelism for NUMA: A uniform abstraction for coordinated scheduling and memory management , 2016, 2016 International Conference on Parallel Architecture and Compilation Techniques (PACT).

[9]  Alexandra Fedorova,et al.  Contention-Aware Scheduling on Multicore Systems , 2010, TOCS.

[10]  Vladimir Vlassov,et al.  Node Architecture Implications for In-Memory Data Analytics on Scale-in Clusters , 2016, 2016 IEEE/ACM 3rd International Conference on Big Data Computing Applications and Technologies (BDCAT).

[11]  Thomas R. Gross,et al.  Memory management in NUMA multicore systems: trapped between cache contention and interconnect overhead , 2011, ISMM '11.

[12]  Carla Schlatter Ellis,et al.  The robustness of NUMA memory management , 1991, SOSP '91.

[13]  Michael J. Franklin,et al.  Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing , 2012, NSDI.

[14]  Li Zhang,et al.  SparkBench: a comprehensive benchmarking suite for in memory data analytic platform Spark , 2015, Conf. Computing Frontiers.

[15]  Norman May,et al.  Scaling Up Concurrent Main-Memory Column-Store Scans: Towards Adaptive NUMA-aware Data and Task Placement , 2015, Proc. VLDB Endow..

[16]  Karine Heydemann,et al.  NUMA-aware scheduling and memory allocation for data-flow task-parallel applications , 2016, PPoPP.

[17]  Christoph Lameter,et al.  NUMA (Non-Uniform Memory Access): An Overview , 2013, ACM Queue.

[18]  Scott Shenker,et al.  Spark: Cluster Computing with Working Sets , 2010, HotCloud.