Evaluating Burst Buffer Placement in HPC Systems

Burst buffers (BBs) are increasingly exploited in contemporary supercomputers to bridge the performance gap between compute and storage systems. The design of BBs, particularly the placement of these devices and the underlying network topology, impacts both performance and cost. As the cost of other components such as memory and accelerators is increasing, it is becoming more important that HPC centers provision BBs tailored to their workloads.This work contributes a provisioning system to provide accurate, multi-tenant simulations that model realistic application and storage workloads from HPC systems. The framework aids HPC centers in modeling their workloads against multiple network and BB configurations rapidly. In experiments with our framework, we provide a comparison of representative Oak Ridge Leadership Computing Facility (OLCF) I/O workloads against multiple BB designs. We analyze the impact of these designs on latency, I/O phase lengths, contention for network and storage devices, and choice of network topology.

[1]  Christopher D. Carothers,et al.  ROSS: a high-performance, low memory, modular time warp system , 2000, PADS '00.

[2]  Christopher D. Carothers,et al.  Parallel distributed simulation and modeling methods: an algorithm for fully-reversible optimistic parallel simulation , 2003, WSC '03.

[3]  Hal Finkel,et al.  HACC: Simulating Sky Surveys on State-of-the-Art Supercomputing Architectures , 2014, 1410.2805.

[4]  Robert B. Ross,et al.  CODES: Enabling Co-Design of Multi-Layer Exascale Storage Architectures , 2011 .

[5]  Robert B. Ross,et al.  Quantifying I/O and Communication Traffic Interference on Dragonfly Networks Equipped with Burst Buffers , 2017, 2017 IEEE International Conference on Cluster Computing (CLUSTER).

[6]  C. Svaneborg.  Large-scale Atomic/Molecular Massively Parallel Simulator , 2011 .

[7]  Maya Gokhale,et al.  Integrated in-system storage architecture for high performance computing , 2012, ROSS '12.

[8]  Teng Wang,et al.  BurstMem: A high-performance burst buffer system for scientific applications , 2014, 2014 IEEE International Conference on Big Data (Big Data).

[9]  Christopher D. Carothers,et al.  Efficient optimistic parallel simulations using reverse computation , 1999, Proceedings Thirteenth Workshop on Parallel and Distributed Simulation. PADS 99. (Cat. No.PR00155).

[10]  Dong H. Ahn,et al.  Scalable I/O-Aware Job Scheduling for Burst Buffer Enabled HPC Clusters , 2016, HPDC.

[11]  William J. Dally,et al.  Technology-Driven, Highly-Scalable Dragonfly Topology , 2008, 2008 International Symposium on Computer Architecture.

[12]  Torsten Hoefler,et al.  Slim Fly: A Cost Effective Low-Diameter Network Topology , 2014, SC14: International Conference for High Performance Computing, Networking, Storage and Analysis.

[13]  Robert M. O'Keefe,et al.  Implementations of time (panel) , 1986, WSC '86.

[14]  Teng Wang,et al.  TRIO: Burst Buffer Based I/O Orchestration , 2015, 2015 IEEE International Conference on Cluster Computing.

[15]  David R. Jefferson,et al.  Virtual time , 1985, ICPP.

[16]  Robert B. Ross,et al.  On the role of burst buffers in leadership-class storage systems , 2012, 012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST).

[17]  Robert Latham,et al.  24/7 Characterization of petascale I/O workloads , 2009, 2009 IEEE International Conference on Cluster Computing and Workshops.

[18]  Jung Ho Ahn,et al.  HyperX: topology, routing, and packaging of efficient large-scale networks , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[19]  Charles E. Leiserson,et al.  Fat-trees: Universal networks for hardware-efficient supercomputing , 1985, IEEE Transactions on Computers.

[20]  Kevin Harms,et al.  Impact of Burst Buffer Architectures on Application Portability , 2016 .

[21]  Peter V. Coveney,et al.  Large-scale molecular dynamics simulation of DNA: implementation and validation of the AMBER98 force field in LAMMPS , 2004, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[22]  Surendra Byna,et al.  Accelerating Science with the NERSC Burst Buffer Early User Program , 2016 .

[23]  Robert B. Ross,et al.  Enabling Parallel Simulation of Large-Scale HPC Network Systems , 2017, IEEE Transactions on Parallel and Distributed Systems.

[24]  R. M. Fujimoto,et al.  Parallel discrete event simulation , 1989, WSC '89.

[25]  Robert B. Ross,et al.  Study of Intra- and Interjob Interference on Torus Networks , 2016, 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS).

[26]  Scott Klasky,et al.  Characterizing output bottlenecks in a supercomputer , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[27]  Lei Cao,et al.  To share or not to share: comparing burst buffer architectures , 2017, SpringSim.

[28]  T. Hahm,et al.  Turbulent Transport Reduction by Zonal Flows: Massively Parallel Simulations , 1998 .

[29]  Bronis R. de Supinski,et al.  The Design, Deployment, and Evaluation of the CORAL Pre-Exascale Systems , 2018, SC18: International Conference for High Performance Computing, Networking, Storage and Analysis.