Extreme-scale scripting: Opportunities for large task-parallel applications on petascale computers

Parallel scripting is a loosely-coupled programming model in which applications are composed of highly parallel scripts of program invocations that process and exchange data via files. We characterize here the applications that can benefit from parallel scripting on petascale-class machines, describe the mechanisms that make this feasible on such systems, and present results achieved with parallel scripts on currently available petascale computers.

[1]  S. Small,et al.  Database-managed grid-enabled analysis of neuroimaging data: the CNARI framework. , 2009, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[2]  Ian T. Foster,et al.  Accelerating Medical Research using the Swift Workflow System , 2007, HealthGrid.

[3]  Yong Zhao,et al.  Many-task computing for grids and supercomputers , 2008, 2008 Workshop on Many-Task Computing on Grids and Supercomputers.

[4]  Glen M. Hocky,et al.  Towards petascale ab initio protein folding through parallel scripting , 2009 .

[5]  Ioan Raicu,et al.  Many-Task Computing: Bridging the Gap between High Throughput Computing and High Performance Computing , 2009 .

[6]  Gregor von Laszewski,et al.  Swift: Fast, Reliable, Loosely Coupled Parallel Computation , 2007, 2007 IEEE Congress on Services (Services 2007).

[7]  Zhao Zhang,et al.  Design and evaluation of a collective IO model for loosely coupled petascale programming , 2008, 2008 Workshop on Many-Task Computing on Grids and Supercomputers.

[8]  Zhao Zhang,et al.  Toward loosely coupled programming on petascale systems , 2008, 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis.

[9]  John K. Ousterhout,et al.  Scripting: Higher-Level Programming for the 21st Century , 1998, Computer.

[10]  Michael C. Neale,et al.  Parallel Workflows for Data-Driven Structural Equation Modeling in Functional Neuroimaging , 2009, Front. Neuroinform..

[11]  Yong Zhao,et al.  A notation and system for expressing and executing cleanly typed workflows on messy scientific data , 2005, SGMD.