Quantifying Scheduling Challenges for Exascale System Software

The move towards high-performance computing (HPC) applications comprised of coupled codes and the need to dramatically reduce data movement is leading to a reexamination of time-sharing vs. space-sharing in HPC systems. In this paper, we discuss and begin to quantify the performance impact of a move away from strict space-sharing of nodes for HPC applications. Specifically, we examine the potential performance cost of time-sharing nodes between application components, we determine whether a simple coordinated scheduling mechanism can address these problems, and we research how suitable simple constraint-based optimization techniques are for solving scheduling challenges in this regime. Our results demonstrate that current general-purpose HPC system software scheduling and resource allocation systems are subject to significant performance deficiencies which we quantify for six representative applications. Based on these results, we discuss areas in which additional research is needed to meet the scheduling challenges of next-generation HPC systems.

[1]  Peter A. Dinda,et al.  VSched: Mixing Batch And Interactive Virtual Machines Using Periodic Real-time Scheduling , 2005, ACM/IEEE SC 2005 Conference (SC'05).

[2]  Alan Burns,et al.  Analysis of Hierarchical EDF Pre-emptive Scheduling , 2007, 28th IEEE International Real-Time Systems Symposium (RTSS 2007).

[3]  Ian T. Foster,et al.  Globus: a Metacomputing Infrastructure Toolkit , 1997, Int. J. High Perform. Comput. Appl..

[4]  Garth A. Gibson,et al.  PRObE: A Thousand-Node Experimental Cluster for Computer Systems Research , 2013, login Usenix Mag..

[5]  Karsten Schwan,et al.  GoldRush: Resource efficient in situ scientific data analytics using fine-grained interference aware execution , 2013, 2013 SC - International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[6]  John Shalf,et al.  Exascale Operating Systems and Runtime Software Report , 2012 .

[7]  Lingjia Tang,et al.  Directly characterizing cross core interference through contention synthesis , 2011, HiPEAC.

[8]  Kun Wang,et al.  Optimizing virtual machine scheduling in NUMA multicore systems , 2013, 2013 IEEE 19th International Symposium on High Performance Computer Architecture (HPCA).

[9]  Alberto Bemporad,et al.  A Matlab function for solving Mixed Integer Quadratic Programs Version 1 . 06 User Guide , 2001 .

[10]  Peter A. Dinda,et al.  Time-Sharing Parallel Applications with Performance Isolation and Control , 2007, Fourth International Conference on Autonomic Computing (ICAC'07).

[11]  Jack Dongarra,et al.  Introduction to the HPCChallenge Benchmark Suite , 2004 .

[12]  Omar U. Pereira Zapata,et al.  EDF and RM Multiprocessor Scheduling Algorithms : Survey and Performance Evaluation , 2005 .

[13]  J. Fier,et al.  Improving the Scalability of Parallel Jobs by adding Parallel Awareness to the Operating System , 2003, ACM/IEEE SC 2003 Conference (SC'03).

[14]  Shinpei Kato,et al.  Gang EDF Scheduling of Parallel Task Systems , 2009, 2009 30th IEEE Real-Time Systems Symposium.

[15]  The CPU Scheduler in VMware vSphere ® 5 , 2013 .

[16]  Sandia Report,et al.  Improving Performance via Mini-applications , 2009 .

[17]  Ron Brightwell,et al.  Characterizing application sensitivity to OS interference using kernel-level noise injection , 2008, 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis.

[18]  Victor Lee,et al.  Implications of I/O for Gang Scheduled Workloads , 1997, JSSPP.

[19]  Peter A. Dinda,et al.  Palacios and Kitten: New high performance operating systems for scalable virtualized and native supercomputing , 2010, 2010 IEEE International Symposium on Parallel & Distributed Processing (IPDPS).

[20]  Sarah Bird.  PACORA : Performance Aware Convex Optimization for Resource Allocation , 2011 .

[21]  Jian Li,et al.  Adjustable Credit Scheduling for High Performance Network Virtualization , 2012, 2012 IEEE International Conference on Cluster Computing.

[22]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[23]  Jiayi Sheng,et al.  A method of quadratic programming for mapping on NoC architecture , 2011, 2011 9th IEEE International Conference on ASIC.

[24]  James W. Layland,et al.  Scheduling Algorithms for Multiprogramming in a Hard-Real-Time Environment , 1989, JACM.

[25]  Daisuke Takahashi,et al.  The HPC Challenge (HPCC) benchmark suite , 2006, SC.

[26]  Rajesh Raman,et al.  Policy driven heterogeneous resource co-allocation with Gangmatching , 2003, High Performance Distributed Computing, 2003. Proceedings. 12th IEEE International Symposium on.

[27]  David E. Bernholdt,et al.  Hobbes: composition and virtualization as the foundations of an extreme-scale OS/R , 2013, ROSS '13.

[28]  Fatma A. Omara,et al.  Genetic algorithms for task scheduling problem , 2010, J. Parallel Distributed Comput..

[29]  Hyong S. Kim,et al.  Is co-scheduling too expensive for SMP VMs? , 2011, EuroSys '11.