Workflow-Level Parameter Study Support for Production Grids

Workflow applications are frequently used in many production Grids. There is a natural need to run the same workflow with many different parameter sets. Unfortunately current Grid portals either do not support this kind of applications or give only specialized support and hence users are obliged to do all the tedious work needed to manage such parameter study applications. P-GRADE portal has been providing a high-level, graphical workflow development and execution environment for various Grids (EGEE, UK NGS, GIN VO, OSG, TeraGrid, etc.) built on second and third generation Grid technologies (GT2, LCG-2, GT4, gLite). Feedback from the user communities of the portal showed that parameter study support is highly needed and hence the recent release of the portal supports the workflow-level parameter study applications. The current paper describes the semantics and implementation principles of managing and executing workflows as parameter studies. Two algorithms are described in detail. The black box algorithm optimizes the usage of storage resources while the PS-labeling algorithm minimizes the load of Grid processing resources. Special emphasis is on the concurrent management of large number of files and jobs in the portal and in the Grids as well as providing a user-friendly, easy-to-use graphical environment to define the workflows and monitor their parametric study execution.

[1]  Maurice Yarrow,et al.  ScyFlow: an environment for the visual specification and execution of scientific workflows , 2006, Concurr. Comput. Pract. Exp..

[2]  Manish Parashar,et al.  Grid Computing — GRID 2002 , 2002, Lecture Notes in Computer Science.

[3]  Francine Berman,et al.  The AppLeS Parameter Sweep Template: User-Level Middleware for the Grid , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[4]  Gábor Terstyánszky,et al.  GEMLCA: Running Legacy Code Applications as Grid Services , 2005, Journal of Grid Computing.

[5]  Rajkumar Buyya,et al.  A Taxonomy of Workflow Management Systems for Grid Computing , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[6]  Francine Berman,et al.  Grid Computing: Making the Global Infrastructure a Reality , 2003 .

[7]  Miron Livny,et al.  Condor and the Grid , 2003 .

[8]  Péter Kacsuk,et al.  Multi-Grid, Multi-User Workflows in the P-GRADE Grid Portal , 2005, Journal of Grid Computing.

[9]  Péter Kacsuk,et al.  Solving the grid interoperability problem by P-GRADE portal at workflow level , 2008, Future Gener. Comput. Syst..

[10]  Henri Casanova,et al.  Parameter Sweeps on the Grid with APST , 2003 .

[11]  Andrew Lewis,et al.  NIMROD/O: A TOOL FOR AUTOMATIC DESIGN OPTIMISATION USING PARALLEL AND DISTRIBUTED SYSTEMS , 2000 .

[12]  Péter Kacsuk,et al.  Grid Meta-Broker Architecture: Towards an Interoperable Grid Resource Brokering Service , 2006, Euro-Par Workshops.

[13]  Rupak Biswas,et al.  An Advanced User Interface Approach for Complex Parameter Study Process Specification on the Information Power Grid , 2000, GRID.