INFORM: integrated flow orchestration and meta-scheduling for managed grid systems

The execution of workflow applications is a reality today in enterprise and scientific grid domains. The core middleware technologies for grids (e.g. meta-schedulers) contain sophisticated resource matching logic, but lack control flow orchestration capability. Workflow orchestrators, on the other hand, suitably control business logic but are unaware of execution requirements of tasks. Marriage of the scheduling technology with workflow management is thereby essential in the design of middleware for geographically distributed grids spanning organizational domains. However, existing endeavors use ad hoc, non-standard solutions, and also lack support for efficient data modeling and handling. In this paper, we present INFORM, an end-to-end middleware solution for grid environments, that integrates workflow orchestration and meta-scheduling. Specific issues explored are job flow modeling, transparent workflow adaptation, and data handling in job flows, through creation of assets using standardized technologies. An implementation of INFORM based on a set of industrial products from IBM is presented and validated using a Montage application. Our results demonstrate significant benefits that can be potentially achieved from such an integrated workflow management approach, when compared to competitive methodologies.