Balancing risk and reward in a market-based task service

We investigate the question of scheduling tasks according to a user-centric value metric-called yield or utility. User value is an attractive basis for allocating shared computing resources, and is fundamental to economic approaches to resource management in linked clusters or grids. Even so, commonly used batch schedulers do not yet support value-based scheduling, and there has been little study of its use in a market-based grid setting. In part this is because scheduling to maximize time-varying value is a difficult problem where even simple formulations are intractable. We present improved heuristics for value-based task scheduling using a simple but rich formulation of value, in which a task's yield decays linearly with its waiting time. We also show the role of value-based scheduling heuristics in a framework for market-based bidding and admission control, in which clients negotiate for task services from multiple grid sites. Our approach follows an investment metaphor: the heuristics balance the risk of future costs against the potential for gains in accepting and scheduling tasks. In particular, we show the importance of opportunity cost, and the impact of risk due to uncertainty in the future job mix.

[1]  Allen B. Downey,et al.  The elusive goal of workload characterization , 1999, PERV.

[2]  Waleed Meleis,et al.  An Experimental Study of Algorithms for Weighted Completion Time Scheduling , 2002, Algorithmica.

[3]  Amin Vahdat,et al.  SHARP: an architecture for secure resource peering , 2003, SOSP '03.

[4]  William E. Weihl,et al.  Lottery scheduling: flexible proportional-share resource management , 1994, OSDI '94.

[5]  Leonid Oliker,et al.  Job Superscheduler Architecture and Performance in Computational Grid Environments , 2003, SC.

[6]  Andrea C. Arpaci-Dusseau,et al.  Explicit Control in the Batch-Aware Distributed File System , 2004, NSDI.

[7]  David E. Irwin,et al.  Dynamic virtual clusters in a grid site manager , 2003, High Performance Distributed Computing, 2003. Proceedings. 12th IEEE International Symposium on.

[8]  T. Kelly Utility-Directed Allocation , 2003 .

[9]  Richard Wolski,et al.  Analyzing Market-Based Resource Allocation Strategies for the Computational Grid , 2001, Int. J. High Perform. Comput. Appl..

[10]  Hussein M. Abdel-Wahab,et al.  A Microeconomic Scheduler for Parallel Computers , 1995, JSSPP.

[11]  David E. Culler,et al.  Market-based cluster resource management , 2001 .

[12]  Amin Vahdat,et al.  Managing energy and server resources in hosting centers , 2001, SOSP.

[13]  Tad Hogg,et al.  Spawn: A Distributed Computational Economy , 1992, IEEE Trans. Software Eng..

[14]  Claudio Bartolini,et al.  Market-Based Resource Allocation for Utility Data Centers , 2003 .

[15]  Jens Mache,et al.  A Comparative Study of Real Workload Traces and Synthetic Workload Models for Parallel Job Scheduling , 1998, JSSPP.

[16]  David E. Culler,et al.  User-Centric Performance Analysis of Market-Based Cluster Batch Schedulers , 2002, 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID'02).

[17]  Alan Burns,et al.  The meaning and role of value in scheduling flexible real-time systems , 2000, J. Syst. Archit..

[18]  David Abramson,et al.  Economic models for resource management and scheduling in Grid computing , 2002, Concurr. Comput. Pract. Exp..

[19]  Michael Stonebraker,et al.  An economic paradigm for query processing and data migration in Mariposa , 1994, Proceedings of 3rd International Conference on Parallel and Distributed Information Systems.

[20]  Ken Chen,et al.  A scheduling algorithm for tasks described by Time Value Function , 1996, Real-Time Systems.

[21]  Ian T. Foster,et al.  SNAP: A Protocol for Negotiating Service Level Agreements and Coordinating Resource Management in Distributed Systems , 2002, JSSPP.

[22]  Waleed Meleis,et al.  Algorithms for total weighted completion time scheduling , 1999, SODA '99.