Large Scale Problem Solving Using Automatic Code Generation and Distributed Visualization

Scientific computation faces multiple scalability challenges in trying to take advantage of the latest generation compute, network and graphics hardware. We present a comprehensive approach to solving four important scalability challenges: programming productivity, scalability to large numbers of processors, I/O bandwidth, and interactive visualization of large data. We describe a scenario where our integrated system is applied in the field of numerical relativity. A solver for the governing Einstein equations is generated and executed on a large computational cluster; the simulation output is distributed onto a distributed data server, and finally visualized using distributed visualization methods and high-speed networks. A demonstration of this system was awarded first place in the IEEE SCALE 2009 Challenge.

[1]  Larry Smarr,et al.  Building an OptIPlanet collaboratory to support microbial metagenomics , 2009, Future Gener. Comput. Syst..

[2]  Christiane Lechner,et al.  From Tensor Equations to Numerical Code - Computer Algebra Tools for Numerical Relativity , 2004, ArXiv.

[3]  Scott H. Hawley,et al.  Evolutions in 3D numerical relativity using fixed mesh refinement , 2003, gr-qc/0310042.

[4]  Brygg Ullmer,et al.  Progressive Retrieval and Hierarchical Visualization of Large Remote Data , 2001, Scalable Comput. Pract. Exp..

[5]  John Shalf,et al.  The Grid and Future Visualization System Architectures , 2003, IEEE Computer Graphics and Applications.

[6]  E. Schnetter,et al.  ALPACA : CACTUS TOOLS FOR APPLICATION LEVEL PERFORMANCE AND CORRECTNESS ANALYSIS , 2008 .

[7]  Chase Qishi Wu,et al.  Ultrascience net: network testbed for large-scale science applications , 2005, IEEE Communications Magazine.

[8]  John Shalf,et al.  Cactus Framework: Black Holes to Gamma Ray Bursts , 2007, ArXiv.

[9]  N. Gehrels,et al.  Gamma-Ray Bursts , 2016, Stars and Stellar Processes.

[10]  S. Sitharama Iyengar,et al.  Optimal pipeline decomposition and adaptive network mapping to support distributed remote visualization , 2007, J. Parallel Distributed Comput..

[11]  B Brügmann,et al.  3D grazing collision of two black holes. , 2001, Physical review letters.

[12]  Brygg Ullmer,et al.  Tangible menus and interaction trays: core tangibles for common physical/digital activities , 2008, TEI.

[13]  Chase Qishi Wu,et al.  CHEETAH: circuit-switched high-speed end-to-end transport architecture testbed , 2005, IEEE Communications Magazine.

[14]  M. Visser Black holes in general relativity , 2009, 0901.4365.

[15]  Petr Holub,et al.  High-definition multimedia for multiparty low-latency interactive communication , 2006, Future Gener. Comput. Syst..

[16]  S. Husa,et al.  Kranc : a Mathematica application to generate numerical codes for tensorial evolution equations , 2004 .

[17]  Gabrielle Allen,et al.  A case study for petascale applications in astrophysics: simulating gamma-ray bursts , 2008, Mardi Gras Conference.

[18]  Renato Pajarola,et al.  Equalizer: A Scalable Parallel Rendering Framework , 2008, IEEE Transactions on Visualization and Computer Graphics.

[19]  John Shalf,et al.  The Cactus Framework and Toolkit: Design and Applications , 2002, VECPAR.

[20]  Ramesh Narayan,et al.  Advection-Dominated Accretion and the Black Hole Event Horizon , 2008, 0803.0322.

[21]  David W. Walker,et al.  RAVE: the resource‐aware visualization environment , 2009, Concurr. Comput. Pract. Exp..

[22]  M. Berger,et al.  Adaptive mesh refinement for hyperbolic partial differential equations , 1982 .

[23]  Andrew A. Chien,et al.  The OptIPuter , 2003, CACM.

[24]  E. Schnetter,et al.  A multi-block infrastructure for three-dimensional time-dependent numerical relativity , 2006, gr-qc/0602104.

[25]  Werner Benger,et al.  Efficient Distributed File I/O for Visualization in Grid Environments , 2000 .

[26]  Larry Smarr,et al.  SPACE‐TIMES GENERATED BY COMPUTERS: BLACK HOLES WITH GRAVITATIONAL RADIATION * , 1977 .

[27]  Joshua R. Smith,et al.  LIGO: The laser interferometer gravitational-wave observatory , 2006, QELS 2006.

[28]  Andy Hopper,et al.  Virtual Network Computing , 1998, IEEE Internet Comput..

[29]  J. Taylor DISCOVERY OF A PULSAR IN A BINARY SYSTEM , 1975 .

[30]  Luc Renambot,et al.  Enabling high resolution collaborative visualization in display rich virtual organizations , 2009, Future Gener. Comput. Syst..

[31]  E. Schnetter,et al.  Turduckening black holes: An Analytical and computational study , 2008, 0809.3533.

[32]  William E. Allcock,et al.  The Globus Striped GridFTP Framework and Server , 2005, ACM/IEEE SC 2005 Conference (SC'05).

[33]  James P. Ahrens,et al.  Remote large data visualization in the paraview framework , 2006, EGPGV '06.

[34]  Werner Benger,et al.  The 3 D Grazing Collision of Two Black Holes , 2002 .

[35]  Kwan-Liu Ma,et al.  Ultra-Scale Visualization: Research and Education , 2007 .

[36]  Venkatram Vishwanath,et al.  Accelerating tropical cyclone analysis using LambdaRAM, a distributed data cache over wide-area ultra-fast networks , 2009, Future Gener. Comput. Syst..

[37]  Robert L. Grossman,et al.  UDT: UDP-based data transfer for high-speed wide area networks , 2007, Comput. Networks.

[38]  Nelson L. Max,et al.  A contract based system for large data visualization , 2005, VIS 05. IEEE Visualization, 2005..

[39]  S. Figuerola,et al.  PHOSPHORUS: single-step on-demand services across multi-domain networks for e-science , 2007, SPIE/OSA/IEEE Asia Communications and Photonics.

[40]  Greg Humphreys,et al.  Chromium: a stream-processing framework for interactive rendering on clusters , 2002, SIGGRAPH.

[41]  Michiaki Hayashi,et al.  G-lambda: Coordination of a Grid Scheduler and Lambda Path Service over GMPLS , 2006, 2006 European Conference on Optical Communications.