HPC s Pivot to Data

Abstract Computer centers such as NERSC and OLCF have traditionally focused on delivering computational capa- bility that enables breakthrough innovation in a wide range of science domains. Accessing that computational power has required services and tools to move the data from input and output to computation and storage. A pivot to data is occurring in HPC. Data transfer tools and services that were previously peripheral are becoming integral to scientific workflows. Emerging requirements from high-bandwidth de- tectors, high-throughput screening techniques, highly concur- rent simulations, increased focus on uncertainty quantification, and an emerging open-data policy posture toward published research are among the data-drivers shaping the networks, file systems, databases, and overall compute and data environment. In this paper we explain the pivot to data in HPC through user requirements and the changing resources provided by HPC with particular focus on data movement. For WAN data transfers we present the results of a study of network performance between centers.

[1]  Thomas F. Jaramillo,et al.  New cubic perovskites for one- and two-photon water splitting using the computational materials repository , 2012 .

[2]  Eirik Endeve,et al.  THE DEVELOPMENT OF EXPLOSIONS IN AXISYMMETRIC AB INITIO CORE-COLLAPSE SUPERNOVA SIMULATIONS OF 12–25 M ⊙ ?> STARS , 2012, 1409.5779.

[3]  Feiyi Wang,et al.  OLCF ’ s 1 TB / s , Next-Generation Lustre File System , 2013 .

[4]  O. E. Bronson Messer,et al.  Near Real-time Data Analysis of Core-collapse Supernova Simulations with Bellerophon , 2014, ICCS.

[5]  Benjamin Teitelbaum,et al.  A One-way Active Measurement Protocol (OWAMP) , 2006, RFC.

[6]  Sin-Jin Lin,et al.  Results from the Daya Bay Reactor Neutrino Experiment , 2014 .

[7]  Michael F. Wehner,et al.  CMIP5 CLIMATE MODEL ANALYSES Climate Extremes in the United States , 2014 .

[8]  Hai Ah Nam,et al.  The practical obstacles of data transfer: why researchers still love scp , 2013, NDM '13.

[9]  Richard P. Mount.  The Office of Science Data-Management Challenge , 2005 .

[10]  Sébastien Boutet,et al.  Simultaneous Femtosecond X-ray Spectroscopy and Diffraction of Photosystem II at Room Temperature , 2013, Science.

[11]  Eli Dart,et al.  The Science DMZ: A network design pattern for data-intensive science , 2013, 2013 SC - International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[12]  Keith T. Knox,et al.  Enhancement of overwritten text in the Archimedes Palimpsest , 2008, Electronic Imaging.

[13]  Scott Klasky,et al.  Terascale direct numerical simulations of turbulent combustion using S3D , 2008 .

[14]  Saul Perlmutter.  Nobel Lecture: Measuring the acceleration of the cosmic expansion using supernovae , 2012 .

[15]  D. Martin Swany,et al.  PerfSONAR: A Service Oriented Architecture for Multi-domain Network Monitoring , 2005, ICSOC.

[16]  P. Jones,et al.  The Twentieth Century Reanalysis Project , 2009 .

[17]  J. T. Childers,et al.  Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC , 2012 .

[18]  M. Sullivan,et al.  PTF 11kx: A Type Ia Supernova with a Symbiotic Nova Progenitor , 2012, Science.