Trigger Happy: Assessing the Viability of Trigger-Based In Situ Analysis

Triggers are an emerging strategy for optimizing execution time for in situ analysis. However, their performance characteristics are complex, making it difficult to decide if a particular trigger-based approach is viable. With this study, we propose a cost model for trigger-based in situ analysis that can assess viability, and we also validate the model's efficacy. Then, once the cost model is established, we apply the model to inform the space of viable approaches, considering variation in simulation code, trigger techniques, and analyses, as well as trigger inspection and fire rates. Real-world values are needed both to validate the model and to use the model to inform the space of viable approaches. We obtain these values by surveying science application teams and by performing runs as large as 2,040 GPUs and 32 billion cells.

[1]  Scott Pakin,et al.  Characterizing and Modeling Power and Energy for Extreme-Scale In-Situ Visualization , 2017, 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[2]  Jeremy S. Meredith,et al.  Parallel in situ coupling of simulation with a fully featured visualization system , 2011, EGPGV '11.

[3]  C. C. Law,et al.  ParaView: An End-User Tool for Large-Data Visualization , 2005, The Visualization Handbook.

[4]  Gunther H. Weber,et al.  Performance Analysis, Design Considerations, and Applications of Extreme-Scale In Situ Infrastructures , 2016, SC16: International Conference for High Performance Computing, Networking, Storage and Analysis.

[5]  Scott Klasky,et al.  Opportunities for Cost Savings with In-Transit Visualization , 2020, ISC.

[6]  Arie Shoshani,et al.  Hello ADIOS: the challenges and lessons of developing leadership class I/O frameworks , 2014, Concurr. Comput. Pract. Exp..

[7]  Stephen A. Jarvis,et al.  CloverLeaf: Preparing Hydrodynamics Codes for Exascale , 2013 .

[8]  James P. Ahrens,et al.  An Image-Based Approach to Extreme Scale in Situ Visualization and Analysis , 2014, SC14: International Conference for High Performance Computing, Networking, Storage and Analysis.

[9]  Bruno Raffin,et al.  High Performance Interactive Computing with FlowVR , 2008 .

[10]  Robert Sisneros,et al.  Damaris/Viz: A nonintrusive, adaptable and user-friendly in situ visualization framework , 2013, 2013 IEEE Symposium on Large-Scale Data Analysis and Visualization (LDAV).

[11]  Karsten Schwan,et al.  GoldRush: Resource efficient in situ scientific data analytics using fine-grained interference aware execution , 2013, 2013 SC - International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[12]  Hank Childs,et al.  VisIt: An End-User Tool for Visualizing and Analyzing Very Large Data , 2011 .

[13]  Utkarsh Ayachit,et al.  ParaView Catalyst: Enabling In Situ Data Analysis and Visualization , 2015, ISAV@SC.

[14]  Katherine Yelick,et al.  Exascale applications: skin in the game , 2020, Philosophical Transactions of the Royal Society A.

[15]  Utkarsh Ayachit,et al.  The SENSEI Generic In Situ Interface , 2016, 2016 Second Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV).

[16]  Sabela Ramos,et al.  Exploring energy-performance-quality tradeoffs for scientific workflows with in-situ data analyses , 2014, Computer Science - Research and Development.

[17]  Guillaume Aupy,et al.  Modeling high-throughput applications for in situ analytics , 2019, Int. J. High Perform. Comput. Appl..

[18]  Scott Klasky,et al.  Comparing the Efficiency of In Situ Visualization Paradigms at Scale , 2019, ISC.

[19]  James P. Ahrens,et al.  ADR visualization: A generalized framework for ranking large-scale scientific data using Analysis-Driven Refinement , 2014, 2014 IEEE 4th Symposium on Large Data Analysis and Visualization (LDAV).

[20]  Scott Klasky,et al.  In Situ Methods, Infrastructures, and Applications on High Performance Computing Platforms , 2016, Comput. Graph. Forum.

[21]  Kwan-Liu Ma,et al.  VTK-m: Accelerating the Visualization Toolkit for Massively Threaded Architectures , 2016, IEEE Computer Graphics and Applications.

[22]  Ron A. Oldfield,et al.  Priority research directions for in situ data management: Enabling scientific discovery from diverse data sources , 2020, Int. J. High Perform. Comput. Appl..

[23]  James P. Ahrens,et al.  The ALPINE In Situ Infrastructure: Ascending from the Ashes of Strawman , 2017, ISAV@SC.

[24]  Hank Childs,et al.  A flexible system for in situ triggers , 2018, ISAV@SC.

[25]  Earl Lawrence,et al.  Partitioning a Large Simulation as It Runs , 2014, Technometrics.

[26]  Matthieu Dreher,et al.  Bredala: Semantic Data Redistribution for In Situ Applications , 2016, 2016 IEEE International Conference on Cluster Computing (CLUSTER).

[27]  N. Anders Petersson,et al.  Toward Exascale Earthquake Ground Motion Simulations for Near-Fault Engineering Analysis , 2017, Computing in Science & Engineering.

[28]  Sven Leyffer,et al.  Optimal scheduling of in-situ analysis for large-scale scientific simulations , 2015, SC15: International Conference for High Performance Computing, Networking, Storage and Analysis.

[29]  Joseph A. Cottam,et al.  A terminology for in situ visualization and analysis systems , 2020, Int. J. High Perform. Comput. Appl..

[30]  Hank Childs,et al.  Benchmarking In Situ Triggers Via Reconstruction Error , 2020 .

[31]  Ali Pinar,et al.  Trigger Detection for Adaptive Scientific Workflows Using Percentile Sampling , 2015, SIAM J. Sci. Comput..

[32]  Michael E. Papka,et al.  Optimal Execution of Co-analysis for Large-Scale Molecular Dynamics Simulations , 2016, SC16: International Conference for High Performance Computing, Networking, Storage and Analysis.

[33]  Hank Childs,et al.  In Situ Visualization for Computational Science , 2019, IEEE Computer Graphics and Applications.

[34]  Ali Pinar,et al.  Enabling Adaptive Scientific Workflows Via Trigger Detection , 2015, ISAV@SC.