Data co processing for extreme scale analysis level ii asc milestone 4745

Exascale supercomputing will embody many revolutionary changes in the hardware and software of high-performance computing. A particularly pressing issue is gaining insight into the science behind the exascale computations. Power and I/O speed con- straints will fundamentally change current visualization and analysis work ows. A traditional post-processing work ow involves storing simulation results to disk and later retrieving them for visualization and data analysis. However, at exascale, scien- tists and analysts will need a range of options for moving data to persistent storage, as the current o ine or post-processing pipelines will not be able to capture the data necessary for data analysis of these extreme scale simulations. This Milestone explores two alternate work ows, characterized as in situ and in transit, and compares them. We nd each to have its own merits and faults, and we provide information to help pick the best option for a particular use.

[1]  Karsten Schwan,et al.  Extending I/O through high performance data services , 2009, 2009 IEEE International Conference on Cluster Computing and Workshops.

[2]  May Concern,et al.  TO WHOM IT MAY CONCERN , 1957, California state journal of medicine.

[3]  Larry Kaplan,et al.  The Gemini System Interconnect , 2010, 2010 18th IEEE Symposium on High Performance Interconnects.

[4]  Seetharami R. Seelam,et al.  Modeling the Impact of Checkpoints on Next-Generation Systems , 2007, 24th IEEE Conference on Mass Storage Systems and Technologies (MSST 2007).

[5]  Utkarsh Ayachit,et al.  The ParaView Guide: A Parallel Visualization Application , 2015 .

[6]  Marianne Winslett,et al.  Server-Directed Collective I/O in Panda , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[7]  Keith D. Underwood,et al.  SeaStar Interconnect: Balanced Bandwidth for Scalable Performance , 2006, IEEE Micro.

[8]  Nathan R. Tallent,et al.  HPCTOOLKIT: tools for performance analysis of optimized parallel programs , 2010, Concurr. Comput. Pract. Exp..

[9]  Analysis of fragmentation in shock physics simulation , 2008, 2008 Workshop on Ultrascale Visualization.

[10]  Karsten Schwan,et al.  DataStager: scalable data staging services for petascale applications , 2009, HPDC '09.

[11]  Rolf Riesen,et al.  CONCURRENCY AND COMPUTATION : PRACTICE AND EXPERIENCE Concurrency Computat , 2008 .

[12]  David Kotz,et al.  Disk-directed I/O for MIMD multiprocessors , 1994, OSDI '94.

[13]  Patrick M. Widener,et al.  Efficient Data-Movement for Lightweight I/O , 2006, 2006 IEEE International Conference on Cluster Computing.

[14]  Abbreviazioni Periodici Giuridici.  C.g.a. , 2013 .

[15]  Kenneth Moreland,et al.  Sandia National Laboratories , 2000 .

[16]  Ron A. Oldfield,et al.  Developing Integrated Data Services for Cray Systems with a Gemini Interconnect. , 2012 .

[17]  Rolf Riesen,et al.  Portals 3.0: protocol building blocks for low overhead communication , 2002, Proceedings 16th International Parallel and Distributed Processing Symposium.

[18]  J. M. McGlaun,et al.  CTH: A software family for multi-dimensional shock physics analysis , 1995 .

[19]  Fenguangzhai Song.  CD , 1992 .

[20]  Sandia Report,et al.  Visualization on Supercomputing Platform Level II ASC Milestone (3537-1B) Results from Sandia , 2010 .

[21]  Kenneth Moreland,et al.  Scalable analysis tools for sensitivity analysis and UQ (3160) results. , 2009 .

[22]  Jay F. Lofstead,et al.  Experiences Applying Data Staging Technology in Unconventional Ways , 2013, 2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing.

[23]  Scott Klasky,et al.  DataSpaces: an interaction and coordination framework for coupled simulation workflows , 2012, HPDC '10.

[24]  Ron A. Oldfield,et al.  Access to External Resources Using Service-Node Proxies , 2009 .

[25]  William Schroeder,et al.  The Visualization Toolkit: An Object-Oriented Approach to 3-D Graphics , 1997 .

[26]  Michael E. Papka,et al.  Topology-aware data movement and staging for I/O acceleration on Blue Gene/P supercomputing systems , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[27]  Scott Klasky,et al.  Examples of in transit visualization , 2011, PDAC '11.

[28]  Ron Oldfield,et al.  Extending scalability of collective IO through nessie and staging , 2011, PDSW '11.

[29]  Kenneth Moreland,et al.  A Survey of Visualization Pipelines , 2013, IEEE Transactions on Visualization and Computer Graphics.

[30]  Ron Oldfield,et al.  Trilinos I/O Support Trios , 2012 .

[31]  Ron A. Oldfield.  Lightweight storage and overlay networks for fault tolerance. , 2006 .

[32]  Rolf Riesen,et al.  Lightweight I/O for Scientific Applications , 2006, 2006 IEEE International Conference on Cluster Computing.

[33]  John P. Morrison,et al.  The Cielo Petascale Capability Supercomputer. , 2011 .

[34]  Nathan Fabian,et al.  In situ fragment detection at scale , 2012, IEEE Symposium on Large Data Analysis and Visualization (LDAV).