Loosely Coupled In Situ Visualization: A Perspective on Why It's Here to Stay

In this position paper, we argue that the loosely coupled in situ processing paradigm will play an important role in high performance computing for the foreseeable future. Loosely coupled in situ is an enabling technique that addresses many of the current issues with tightly coupled in situ, including, ease-of-integration, usability, and fault tolerance. We survey the prominent positives and negatives of both tightly coupled and loosely coupled in situ and present our recommendation as to why loosely coupled in situ is an enabling technique that is here to stay. We then report on some recent experiences with loosely coupled in situ processing, in an effort to explore each of the discussed factors in a real-world environment.

[1]  Kwan-Liu Ma,et al.  Dax Toolkit: A proposed framework for data analysis and visualization at Extreme Scale , 2011, 2011 IEEE Symposium on Large Data Analysis and Visualization.

[2]  Prabhat,et al.  Extreme Scaling of Production Visualization Software on Diverse Architectures , 2010, IEEE Computer Graphics and Applications.

[3]  Kesheng Wu,et al.  Scientific Discovery at the Exascale , 2011 .

[4]  James P. Ahrens,et al.  PISTON: A Portable Cross-Platform Framework for Data-Parallel Visualization Operators , 2012, EGPGV@Eurographics.

[5]  Fan Zhang,et al.  ActiveSpaces: Exploring dynamic code deployment for extreme scale data processing , 2015, Concurr. Comput. Pract. Exp..

[6]  Arie Shoshani,et al.  Hello ADIOS: the challenges and lessons of developing leadership class I/O frameworks , 2014, Concurr. Comput. Pract. Exp..

[7]  Scott Klasky,et al.  Towards Scalable Visualization Plugins for Data Staging Workflows , 2015 .

[8]  Nagiza F. Samatova,et al.  Compressed ion temperature gradient turbulence in diverted tokamak edge , 2009 .

[9]  Robert Sisneros,et al.  EAVL: The Extreme-scale Analysis and Visualization Library , 2012, EGPGV@Eurographics.

[10]  Scott Klasky,et al.  Examples of in transit visualization , 2011, PDAC '11.

[11]  Gunther H. Weber,et al.  Visualization at Extreme Scale Concurrency , 2012, High Performance Visualization.

[12]  Fan Zhang,et al.  Combining in-situ and in-transit processing to enable extreme-scale scientific analysis , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[13]  Kenneth Moreland,et al.  Sandia National Laboratories , 2000 .

[14]  Karsten Schwan,et al.  Flexible IO and integration for scientific codes through the adaptable IO system (ADIOS) , 2008, CLADE '08.

[15]  Michael E. Papka,et al.  Toward simulation-time data analysis and I/O acceleration on leadership-class systems , 2011, 2011 IEEE Symposium on Large Data Analysis and Visualization.

[16]  Jeremy S. Meredith,et al.  Parallel in situ coupling of simulation with a fully featured visualization system , 2011, EGPGV '11.

[17]  James P. Ahrens,et al.  The SDAV Software Frameworks for Visualization and Analysis on Next-Generation Multi-Core and Many-Core Architectures , 2012, 2012 SC Companion: High Performance Computing, Networking Storage and Analysis.

[18]  Robert Sisneros,et al.  A distributed data-parallel framework for analysis and visualization algorithm development , 2012, GPGPU-5.

[19]  E. Wes Bethel,et al.  High Performance Visualization - Enabling Extreme-Scale Scientific Insight , 2012, High Performance Visualization.

[20]  Fan Zhang,et al.  In Situ Processing , 2012, High Performance Visualization.

[21]  Scott Klasky,et al.  DataSpaces: an interaction and coordination framework for coupled simulation workflows , 2012, HPDC '10.