A Maya use case: adaptable scientific workflows with ADIOS for general relativistic astrophysics

There are many challenges in analyzing and visualizing data from current cutting-edge general relativistic astrophysics simulations. Many of the associated tasks are time-consuming with large performance degradation due to the magnitude and complexity of the data. The Adaptable I/O System (ADIOS) is a componentization of the I/O layer that has demonstrated remarkable I/O performance improvements on applications running on leadership class machines while also offering new in-memory "staging" operations for transforming data in-situ. We have been incorporating ADIOS staging technologies into our Maya numerical relativity code based on Cactus infrastructure and Carpet mesh refinement. Incorporating ADIOS into the Maya code is the first step toward enabling a more adaptable Maya workflow. We provide descriptions how we intend to leverage FlexPath (an ADIOS transport method) to provide a richer user experience in real-time visualization and interactive steering.

[1]  Karsten Schwan,et al.  Adaptable, metadata rich IO methods for portable high performance IO , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[2]  Jeremy S. Meredith,et al.  Parallel in situ coupling of simulation with a fully featured visualization system , 2011, EGPGV '11.

[3]  John Biddiscombe,et al.  Parallel Computational Steering for HPC Applications Using HDF5 Files in Distributed Shared Memory , 2012, IEEE Transactions on Visualization and Computer Graphics.

[4]  Kwan-Liu Ma,et al.  A preview and exploratory technique for large-scale scientific simulations , 2011, EGPGV '11.

[5]  Mark L. Sawley,et al.  Distributed run-time visualization and solution steering of parallel flow simulations , 1998 .

[6]  C. Ott,et al.  The Einstein Toolkit: a community computational infrastructure for relativistic astrophysics , 2011, 1111.3344.

[7]  Ray W. Grout,et al.  Ultrascale Visualization In Situ Visualization for Large-Scale Combustion Simulations , 2010 .

[8]  Karsten Schwan,et al.  Falcon: on-line monitoring and steering of large-scale parallel programs , 1995, Proceedings Frontiers '95. The Fifth Symposium on the Frontiers of Massively Parallel Computation.

[9]  Allan Tuchman,et al.  A System for Remote Data Visualization , 2007 .

[10]  John Shalf,et al.  The Cactus Framework and Toolkit: Design and Applications , 2002, VECPAR.

[11]  Akira Kageyama,et al.  An approach to exascale visualization: Interactive viewing of in-situ visualization , 2013, Comput. Phys. Commun..

[12]  Stanley Deser,et al.  Dynamical Structure and Definition of Energy in General Relativity , 1959 .

[13]  Karsten Schwan,et al.  Event-based systems: opportunities and challenges at exascale , 2009, DEBS '09.

[14]  James P. Ahrens,et al.  Remote large data visualization in the paraview framework , 2006, EGPGV '06.

[15]  Karsten Schwan,et al.  Flexible IO and integration for scientific codes through the adaptable IO system (ADIOS) , 2008, CLADE '08.

[16]  Department of Physics,et al.  WhiskyMHD: a new numerical code for general relativistic magnetohydrodynamics , 2007, gr-qc/0701109.

[17]  S. Husa,et al.  Kranc : a Mathematica application to generate numerical codes for tensorial evolution equations , 2004 .

[18]  James Arthur Kohl,et al.  Cumulvs: Providing Fault Toler. Ance, Visualization, and Steer Ing of Parallel Applications , 1996, Int. J. High Perform. Comput. Appl..

[19]  Christiane Lechner,et al.  Kranc: a Mathematica package to generate numerical codes for tensorial evolution equations , 2006, Comput. Phys. Commun..

[20]  Fan Zhang,et al.  Combining in-situ and in-transit processing to enable extreme-scale scientific analysis , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[21]  Karsten Schwan,et al.  FlexIO: I/O Middleware for Location-Flexible Scientific Data Analytics , 2013, 2013 IEEE 27th International Symposium on Parallel and Distributed Processing.

[22]  Scott H. Hawley,et al.  Evolutions in 3D numerical relativity using fixed mesh refinement , 2003, gr-qc/0310042.

[23]  C.R. Johnson,et al.  SCIRun: A Scientific Programming Environment for Computational Steering , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[24]  J. D. Brunner,et al.  VASE: the visualization and application steering environment , 1993, Supercomputing '93.

[25]  David R. O'Hallaron,et al.  Remote runtime steering of integrated terascale simulation and visualization , 2006, SC.