Programmable In Situ System for Iterative Workflows

We describe an in situ system for solving iterative problems. We specifically target inverse problems, where expensive simulations are approximated using a surrogate model. The model explores the parameter space of the simulation through iterative trials, each of which becomes a job managed by a parallel scheduler. Our work extends Henson [1], a cooperative multi-tasking system for in situ execution of loosely coupled codes.

[1]  Arie Shoshani,et al.  Hello ADIOS: the challenges and lessons of developing leadership class I/O frameworks , 2014, Concurr. Comput. Pract. Exp..

[2]  Fan Zhang,et al.  Adaptive data placement for staging-based coupled scientific workflows , 2015, SC15: International Conference for High Performance Computing, Networking, Storage and Analysis.

[3]  Jean M. Sexton,et al.  Nyx: A MASSIVELY PARALLEL AMR CODE FOR COMPUTATIONAL COSMOLOGY , 2013, J. Open Source Softw..

[4]  Jeremy S. Meredith,et al.  Parallel in situ coupling of simulation with a fully featured visualization system , 2011, EGPGV '11.

[5]  Christine A. Shoemaker,et al.  A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions , 2007, INFORMS J. Comput..

[6]  Gunther H. Weber,et al.  Performance Analysis, Design Considerations, and Applications of Extreme-Scale In Situ Infrastructures , 2016, SC16: International Conference for High Performance Computing, Networking, Storage and Analysis.

[7]  Michael E. Papka,et al.  Topology-aware data movement and staging for I/O acceleration on Blue Gene/P supercomputing systems , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[8]  Daniel S. Katz,et al.  Swift/T: Large-Scale Application Composition via Distributed-Memory Dataflow Processing , 2013, 2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing.

[9]  Hans-Martin Gutmann,et al.  A Radial Basis Function Method for Global Optimization , 2001, J. Glob. Optim..

[10]  G. Gary Wang,et al.  Review of Metamodeling Techniques in Support of Engineering Design Optimization , 2007, DAC 2006.

[11]  Sriram Krishnamoorthy,et al.  Noncollective Communicator Creation in MPI , 2011, EuroMPI.

[12]  Dmitriy Morozov,et al.  Master of Puppets: Cooperative Multitasking for In Situ Processing , 2016, HPDC.

[13]  Christine A. Shoemaker,et al.  Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems , 2014, J. Glob. Optim..

[14]  M. Viel,et al.  Warm dark matter as a solution to the small scale crisis: New constraints from high redshift Lyman-α forest data , 2013, 1306.2314.

[15]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .

[16]  Robert Sisneros,et al.  Damaris/Viz: A nonintrusive, adaptable and user-friendly in situ visualization framework , 2013, 2013 IEEE Symposium on Large-Scale Data Analysis and Visualization (LDAV).

[17]  M. White,et al.  The Lyman α forest in optically thin hydrodynamical simulations , 2014, 1406.6361.

[18]  Justin M. Wozniak,et al.  Lessons Learned from Building In Situ Coupling Frameworks , 2015, ISAV@SC.