A taxonomy of market‐based resource management systems for utility‐driven cluster computing

In utility‐driven cluster computing, cluster Resource Management Systems (RMSs) need to know the specific needs of different users in order to allocate resources according to their needs. This in turn is vital to achieve service‐oriented Grid computing that harnesses resources distributed worldwide based on users' objectives. Recently, numerous market‐based RMSs have been proposed to make use of real‐world market concepts and behavior to assign resources to users for various computing platforms. The aim of this paper is to develop a taxonomy that characterizes and classifies how market‐based RMSs can support utility‐driven cluster computing in practice. The taxonomy is then mapped to existing market‐based RMSs designed for both cluster and other computing platforms to survey current research developments and identify outstanding issues. Copyright © 2006 John Wiley & Sons, Ltd.

[1]  Rajkumar Buyya,et al.  Pricing for Utility-Driven Resource Management and Allocation in Clusters , 2007, Int. J. High Perform. Comput. Appl..

[2]  Rajkumar Buyya,et al.  Peer-to-Peer Grid Computing and a .NET-Based Alchemi Framework , 2006 .

[3]  Rajkumar Buyya,et al.  A Case for Cooperative and Incentive-Based Coupling of Distributed Clusters , 2005, 2005 IEEE International Conference on Cluster Computing.

[4]  Rajkumar Buyya,et al.  Service Level Agreement based Allocation of Cluster Resources: Handling Penalty to Enhance Utility , 2005, 2005 IEEE International Conference on Cluster Computing.

[5]  D. Rus,et al.  Utility Driven Mobile-Agent Scheduling , 2005 .

[6]  David Abramson,et al.  The Grid Economy , 2005, Proceedings of the IEEE.

[7]  Rajkumar Buyya,et al.  A novel architecture for realizing grid workflow using tuple spaces , 2004, Fifth IEEE/ACM International Workshop on Grid Computing.

[8]  Matjaz B. Juric,et al.  Business process execution language for web services , 2004 .

[9]  L. Kalé,et al.  Faucets: efficient resource allocation on the computational grid , 2004, International Conference on Parallel Processing, 2004. ICPP 2004..

[10]  David E. Irwin,et al.  Balancing risk and reward in a market-based task service , 2004, Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004..

[11]  Rajkumar Buyya,et al.  A grid service broker for scheduling distributed data-oriented applications on global grids , 2004, MGC '04.

[12]  Rajkumar Buyya,et al.  Libra: a computational economy‐based job scheduling system for clusters , 2004, Softw. Pract. Exp..

[13]  Bernardo A. Huberman,et al.  Tycoon: A Market-Based Resource Allocation System , 2004, ArXiv.

[14]  Guangwen Yang,et al.  Gridmarket: A Practical, Efficient Market Balancing Resource for Grid and P2P Computing , 2003, GCC.

[15]  Cyrus Harrison,et al.  OCEAN: the open computation exchange and arbitration network, a market approach to meta computing , 2003, Second International Symposium on Parallel and Distributed Computing, 2003. Proceedings..

[16]  David E. Irwin,et al.  Dynamic virtual clusters in a grid site manager , 2003, High Performance Distributed Computing, 2003. Proceedings. 12th IEEE International Symposium on.

[17]  Torsten Eymann,et al.  Decentralized resource allocation in application layer networks , 2003, CCGrid 2003. 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid, 2003. Proceedings..

[18]  Heiko Ludwig,et al.  The WSLA Framework: Specifying and Monitoring Service Level Agreements for Web Services , 2003, Journal of Network and Systems Management.

[19]  David Abramson,et al.  Economic models for resource management and scheduling in Grid computing , 2002, Concurr. Comput. Pract. Exp..

[20]  Rajkumar Buyya,et al.  GridBank: a Grid Accounting Services Architecture (GASA) for distributed systems sharing and integration , 2002, Proceedings International Parallel and Distributed Processing Symposium.

[21]  F. Berman.  The Teragrid Project , 2002 .

[22]  Hector Garcia-Molina,et al.  Bidding for storage space in a peer-to-peer data preservation system , 2002, Proceedings 22nd International Conference on Distributed Computing Systems.

[23]  Laxmikant V. Kalé,et al.  A Malleable-Job System for Timeshared Parallel Machines , 2002, 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID'02).

[24]  David E. Culler,et al.  User-Centric Performance Analysis of Market-Based Cluster Batch Schedulers , 2002, 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID'02).

[25]  Muthucumaru Maheswaran,et al.  Supporting co-allocation in an auctioning-based resource allocator for grid systems , 2002, Proceedings 16th International Parallel and Distributed Processing Symposium.

[26]  Rajkumar Buyya,et al.  A taxonomy and survey of grid resource management systems for distributed computing , 2002, Softw. Pract. Exp..

[27]  David Abramson,et al.  A Computational Economy for Grid Computing and its Implementation in the Nimrod-G Resource Brok , 2001, Future Gener. Comput. Syst..

[28]  Richard Wolski,et al.  Analyzing Market-Based Resource Allocation Strategies for the Computational Grid , 2001, Int. J. High Perform. Comput. Appl..

[29]  Laxmikant V. Kalé,et al.  Adaptive Load Balancing for MPI Programs , 2001, International Conference on Computational Science.

[30]  Claudio Bartolini,et al.  Economic dynamics of agents in multiple auctions , 2001, AGENTS '01.

[31]  Hai Jin,et al.  Single System Image , 2001, Int. J. High Perform. Comput. Appl..

[32]  David Abramson,et al.  A case for economy grid architecture for service oriented grid computing , 2001, Proceedings 15th International Parallel and Distributed Processing Symposium. IPDPS 2001.

[33]  Spyros Lalis,et al.  JaWS: An Open Market-Based Framework for Distributed Computing over the Internet , 2000, GRID.

[34]  Baruch Awerbuch,et al.  An Opportunity Cost Approach for Job Assignment in a Scalable Computing Cluster , 2000, IEEE Trans. Parallel Distributed Syst..

[35]  David Abramson,et al.  Nimrod/G: an architecture for a resource management and scheduling system in a global computational grid , 2000, Proceedings Fourth International Conference/Exhibition on High Performance Computing in the Asia-Pacific Region.

[36]  Yechiam Yemini,et al.  MarketNet: protecting access to information systems through financial market controls , 2000, Decis. Support Syst..

[37]  David E. Culler,et al.  Market-based Proportional Resource Sharing for Clusters , 2000 .

[38]  P. Varaiya,et al.  Providing Internet access: what we learn from INDEX , 1999, IEEE Netw..

[39]  Rajkumar Buyya,et al.  High Performance Cluster Computing: Architectures and Systems , 1999 .

[40]  Ian Pratt,et al.  Xenoservers: accountable execution of untrusted programs , 1999, Proceedings of the Seventh Workshop on Hot Topics in Operating Systems.

[41]  S. Kapidakis,et al.  A computational economy for dynamic load balancing and data replication , 1998, ICE '98.

[42]  N. Nisan,et al.  The POPCORN market—an online market for computational resources , 1998, ICE '98.

[43]  Baruch Awerbuch,et al.  A cost-benefit framework for online management of a metacomputing system , 1998, ICE '98.

[44]  Bin Yao,et al.  A taxonomy for describing matching and scheduling heuristics for mixed-machine heterogeneous computing systems , 1998, Proceedings Seventeenth IEEE Symposium on Reliable Distributed Systems (Cat. No.98CB36281).

[45]  Amnon Barak,et al.  The MOSIX multicomputer operating system for high performance cluster computing , 1998, Future Gener. Comput. Syst..

[46]  Alexander Pfaffinger,et al.  Economic-Based Dynamic Load Distribution in Large Workstation Networks , 1996, Euro-Par, Vol. II.

[47]  Veljko Milutinovic,et al.  A survey of heterogeneous computing: concepts and systems , 1996, Proc. IEEE.

[48]  Veljko M. Milutinovic,et al.  EM³: A Taxonomy of Heterogeneous Computing Systems , 1995, Computer.

[49]  David Abramson,et al.  Nimrod: a tool for performing parametrised simulations using distributed workstations , 1995, Proceedings of the Fourth IEEE International Symposium on High Performance Distributed Computing.

[50]  Hussein M. Abdel-Wahab,et al.  A Microeconomic Scheduler for Parallel Computers , 1995, JSSPP.

[51]  David D. Clark,et al.  A Model for Cost Allocation and Pricing in the Internet , 1995 .

[52]  Michael Stonebraker,et al.  An economic paradigm for query processing and data migration in Mariposa , 1994, Proceedings of 3rd International Conference on Parallel and Distributed Information Systems.

[53]  Michael P. Wellman.  A Market-Oriented Programming Environment and its Application to Distributed Multicommodity Flow Problems , 1993, J. Artif. Intell. Res..

[54]  Tad Hogg,et al.  Spawn: A Distributed Computational Economy , 1992, IEEE Trans. Software Eng..

[55]  Deborah Estrin,et al.  A study of priority pricing in multiple service class networks , 1991, SIGCOMM '91.

[56]  Rahul Simha,et al.  A Microeconomic Approach to Optimal Resource Allocation in Distributed Computer Systems , 1989, IEEE Trans. Computers.

[57]  Donald F. Ferguson,et al.  Microeconomic algorithms for load balancing in distributed computer systems , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[58]  Thomas L. Casavant,et al.  A Taxonomy of Scheduling in General-Purpose Distributed Computing Systems , 1988, IEEE Trans. Software Eng..

[59]  P. Altena,et al.  In search of clusters , 2007 .

[60]  Daniel Grosu,et al.  AUCTION-BASED RESOURCE ALLOCATION PROTOCOLS IN GRIDS , 2004 .

[61]  Amin Vahdat,et al.  Resource Allocation in Federated Distributed Computing Infrastructures , 2004 .

[62]  S. Tuecke,et al.  Web services agreement specification (ws-agreement) , 2004 .

[63]  Claudio Bartolini,et al.  Market-Based Resource Allocation for Utility Data Centers , 2003 .

[64]  William Gropp,et al.  Beowulf Cluster Computing with Linux , 2003 .

[65]  R. V. van Nieuwpoort,et al.  The Grid 2: Blueprint for a New Computing Infrastructure , 2003 .

[66]  Tony Andrews,et al.  Business Process Execution Language for Web Services Version 1.1 , 2003 .

[67]  Baruch Awerbuch,et al.  An Opportunity Cost Approach for Job Assignment and Reassignment in a Scalable Computing Cluster , 2002 .

[68]  David Abramson,et al.  An Evaluation of Economy-based Resource Trading and Scheduling on Computational Power Grids for Parameter Sweep Applications , 2000 .

[69]  Gregory Francis Pfister,et al.  In search of clusters (2nd ed.) , 1998 .

[70]  Chris J. Scheiman,et al.  SuperWeb: research issues in Java-based global computing , 1997, Concurr. Pract. Exp..

[71]  H. G. Rotithor.  Taxonomy of dynamic task scheduling schemes in distributed computing systems , 1994 .

[72]  E. Drexler,et al.  Incentive engineering for computational resource management , 1988 .

[73]  R. Buyya,et al.  A taxonomy and survey of grid resource management systems for distributed computing , 2022 .