Performance Analysis, Design Considerations, and Applications of Extreme-Scale In Situ Infrastructures

A key trend facing extreme-scale computational science is the widening gap between computational and I/O rates, and the challenge that follows is how to best gain insight from simulation data when it is increasingly impractical to save it to persistent storage for subsequent visual exploration and analysis. One approach to this challenge is centered around the idea of in situ processing, where visualization and analysis processing is performed while data is still resident in memory. This paper examines several key design and performance issues related to the idea of in situ processing at extreme scale on modern platforms: scalability, overhead, performance measurement and analysis, comparison and contrast with a traditional post hoc approach, and interfacing with simulation codes. We illustrate these principles in practice with studies, conducted on large-scale HPC platforms, that include a miniapplication and multiple science application codes, one of which demonstrates in situ methods in use at greater than 1M-way concurrency.

[1]  Richard W. Hamming,et al.  Numerical Methods for Scientists and Engineers , 1962 .

[2]  E. E. Zajac.  Computer-made perspective movies as a scientific and communication tool , 1964, CACM.

[3]  S. Orszag,et al.  Secondary instability of a temporally growing mixing layer , 1987, Journal of Fluid Mechanics.

[4]  David H. Laidlaw,et al.  The application visualization system: a computational environment for scientific visualization , 1989, IEEE Computer Graphics and Applications.

[5]  Robert Haimes,et al.  pV3 - A distributed system for large-scale unsteady CFD visualization , 1994 .

[6]  Al Globus.  A software model for visualization of large unsteady 3-D CFD results , 1995 .

[7]  T. Smith,et al.  The Structure of Premixed Flames in a Spatially Evolving Turbulent Flow , 1996 .

[8]  James Arthur Kohl,et al.  Cumulvs: Providing Fault Toler. Ance, Visualization, and Steer Ing of Parallel Applications , 1996, Int. J. High Perform. Comput. Appl..

[9]  Jarke J. van Wijk,et al.  A survey of computational steering environments , 1999, Future Gener. Comput. Syst..

[10]  K. Jansen.  A stabilized finite element method for computing turbulence , 1999 .

[11]  Kenneth E. Jansen,et al.  A stabilized finite element method for the incompressible Navier–Stokes equations using a hierarchical basis , 2001 .

[12]  John Shalf,et al.  The Cactus Framework and Toolkit: Design and Applications , 2002, VECPAR.

[13]  S. Dey,et al.  Hierarchical basis for stabilized finite element methods for compressible flows , 2003 .

[14]  S. Menon,et al.  Open-loop control of combustion instabilities in a model gas turbine combustor , 2003 .

[15]  K. Jansen,et al.  A dynamic Smagorinsky model with dynamic determination of the filter width ratio , 2004 .

[16]  John L. Henning.  SPEC CPU2006 benchmark descriptions , 2006, CARN.

[17]  Charles A. Taylor,et al.  Outflow boundary conditions for three-dimensional finite element modeling of blood flow and pressure in arteries , 2006 .

[18]  Cloyce D. Spradling.  SPEC CPU2006 benchmark tools , 2007, CARN.

[19]  Charles A. Taylor,et al.  Augmented Lagrangian method for constraining the shape of velocity profiles at outlet boundaries for three-dimensional finite element simulations of blood flow , 2009 .

[20]  Karsten Schwan,et al.  Managing Variability in the IO Performance of Petascale Storage Systems , 2010, 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis.

[21]  Jeremy S. Meredith,et al.  Parallel in situ coupling of simulation with a fully featured visualization system , 2011, EGPGV '11.

[22]  K. Jansen,et al.  Three-dimensional interactions between a finite-span synthetic jet and a crossflow , 2011, Journal of Fluid Mechanics.

[23]  Michael E. Papka,et al.  Topology-aware data movement and staging for I/O acceleration on Blue Gene/P supercomputing systems , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[24]  E. Wes Bethel,et al.  Hybrid Parallelism for Volume Rendering on Large-, Multi-, and Many-Core Systems , 2012, IEEE Transactions on Visualization and Computer Graphics.

[25]  Fan Zhang,et al.  In Situ Processing , 2012, High Performance Visualization.

[26]  Karsten Schwan,et al.  GoldRush: Resource efficient in situ scientific data analytics using fine-grained interference aware execution , 2013, 2013 SC - International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[27]  Robert Sisneros,et al.  Damaris/Viz: A nonintrusive, adaptable and user-friendly in situ visualization framework , 2013, 2013 IEEE Symposium on Large-Scale Data Analysis and Visualization (LDAV).

[28]  In Situ Pathtube Visualization with Explorable Images , 2013, EGPGV@Eurographics.

[29]  Onkar Sahni,et al.  A parallel adaptive mesh method for the numerical simulation of multiphase flows , 2013 .

[30]  Arie Shoshani,et al.  Hello ADIOS: the challenges and lessons of developing leadership class I/O frameworks , 2014, Concurr. Comput. Pract. Exp..

[31]  Onkar Sahni,et al.  Scalable Implicit Flow Solver for Realistic Wing Simulations with Flow Control , 2014, Computing in Science & Engineering.

[32]  Benjamin A. Matthews,et al.  Scalable fully implicit finite element flow solver with application to high-fidelity flow control simulations on a realistic wing design , 2014 .

[33]  James P. Ahrens,et al.  An Image-Based Approach to Extreme Scale in Situ Visualization and Analysis , 2014, SC14: International Conference for High Performance Computing, Networking, Storage and Analysis.

[34]  K. Moreland,et al.  Instruction Memory Overhead of In Situ Visualization and Analysis Libraries on HPC Machines , 2014 .

[35]  Jens H. Krüger,et al.  Freeprocessing: Transparent in situ Visualization via Data Interception , 2014, EGPGV@EuroVis.

[36]  Karsten Schwan,et al.  Flexpath: Type-Based Publish/Subscribe System for Large-Scale Science Analytics , 2014, 2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[37]  Hank Childs,et al.  Strawman: A Batch In Situ Visualization and Analysis Infrastructure for Multi-Physics Simulation Codes , 2015, ISAV@SC.

[38]  Karsten Schwan,et al.  SODA: Science-Driven Orchestration of Data Analytics , 2015, 2015 IEEE 11th International Conference on e-Science.

[39]  K. Jansen,et al.  A study on large bubble motion and liquid film in vertical pipes and inclined narrow channels , 2015 .

[40]  M. White,et al.  The Lyman α forest in optically thin hydrodynamical simulations , 2014, 1406.6361.

[41]  Y. Elimelech,et al.  Experimental and numerical investigation on steady blowing flow control within a compact inlet duct , 2015 .

[42]  The In Situ Terminology Project , 2016 .

[43]  Scott Klasky,et al.  In Situ Methods, Infrastructures, and Applications on High Performance Computing Platforms , 2016, Comput. Graph. Forum.