Applying workflow as a service paradigm to application farming

Task farming is often used to enable parameter sweep for exploration of large sets of initial conditions for large scale complex simulations. Such applications occur very often in life sciences. Available solutions enable to perform parameter sweep by creating multiple job submissions with different parameters. This paper presents an approach to farm workflows, employing service oriented paradigms using the WS‐VLAM workflow manager, which provides ways to create, control, and monitor workflow applications and their components. We present two service‐oriented approaches for workflow farming: task level, whereby task harness acts as services by being invoked on which task to load, and data level, where the actual task is invoked as a service with different chunks of data to process. An experimental evaluation of the presented solution is performed with a biomedical application for which 3000 simulations were required to perform a Monte Carlo study. Copyright © 2013 John Wiley & Sons, Ltd.

[1]  Carole A. Goble,et al.  Scientific Workflows as Services in caGrid: A Taverna and gRAVI Approach , 2009, 2009 IEEE International Conference on Web Services.

[2]  Luc Moreau,et al.  The Open Provenance Model: An Overview , 2008, IPAW.

[3]  Rajkumar Buyya,et al.  Workflow Engine for Clouds , 2011, CloudCom 2011.

[4]  Louis O. Hertzberger,et al.  VLAM-G: Interactive data driven workflow engine for Grid-enabled resources , 2007, Sci. Program..

[5]  Satoshi Matsuoka,et al.  Ninf-G: A Reference Implementation of RPC-based Programming Middleware for Grid Computing , 2003, Journal of Grid Computing.

[6]  John Shalf,et al.  Classifying and enabling Grid Applications , 2003 .

[7]  Henri Casanova,et al.  Parameter Sweeps on the Grid with APST , 2003 .

[8]  Roger M. Cooke,et al.  Techniques for generic probabilistic inversion , 2006, Comput. Stat. Data Anal..

[9]  Jack J. Dongarra,et al.  GridSolve: The Evolution of A Network Enabled Solver , 2006, Grid-Based Problem Solving Environments.

[10]  Marian Bubak,et al.  Workflow as a service: an approach to workflow farming , 2012, ECMLS '12.

[11]  Scott Klasky,et al.  Scientific Process Automation and Workflow Management , 2009 .

[12]  Warren Smith,et al.  Scheduling with advanced reservations , 2000, Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000.

[13]  W. Huberts,et al.  Personalized computational modeling of vascular access creation , 2012 .

[14]  Francine Berman,et al.  The AppLeS Parameter Sweep Template: User-Level Middleware for the Grid , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[15]  Johan Tordsson,et al.  Three fundamental dimensions of scientific workflow interoperability: Model of computation, language, and execution environment , 2010, Future Gener. Comput. Syst..

[16]  Adam Belloum,et al.  HisT/PLIER: A Two-Fold Provenance Approach for Grid-Enabled Scientific Workflows Using WS-VLAM , 2011, 2011 IEEE/ACM 12th International Conference on Grid Computing.

[17]  Valeria V. Krzhizhanovskaya,et al.  Dynamic workload balancing of parallel applications with user-level scheduling on the Grid , 2009, Future Gener. Comput. Syst..

[18]  J. Moscicki.  Understanding and Mastering Dynamics in Computing Grids , 2013 .

[19]  Marian Bubak,et al.  Prediction-based auto-scaling of scientific workflows , 2011, MGC '11.

[20]  Jianwu Wang,et al.  Accelerating Parameter Sweep Workflows by Utilizing Ad-hoc Network Computing Resources: An Ecological Example , 2009, 2009 Congress on Services - I.

[21]  David P. Anderson,et al.  BOINC: a system for public-resource computing and storage , 2004, Fifth IEEE/ACM International Workshop on Grid Computing.

[22]  David Abramson,et al.  A Computational Economy for Grid Computing and its Implementation in the Nimrod-G Resource Brok , 2001, Future Gener. Comput. Syst..

[23]  F N van de Vosse,et al.  Estimation of distributed arterial mechanical properties using a wave propagation model in a reverse way. , 2010, Medical engineering & physics.