The Grid Economy

This work identifies challenges in managing resources in a Grid computing environment and proposes computational economy as a metaphor for effective management of resources and application scheduling. It identifies distributed resource management challenges and requirements of economy-based Grid systems, and discusses various representative economy-based systems, both historical and emerging, for cooperative and competitive trading of resources such as CPU cycles, storage, and network bandwidth. It presents an extensive, service-oriented Grid architecture driven by Grid economy and an approach for its realization by leveraging various existing Grid technologies. It also presents commodity and auction models for resource allocation. The use of commodity economy model for resource management and application scheduling in both computational and data grids is also presented.

[1]  R. F. Freund,et al.  Dynamic Mapping of a Class of Independent Tasks onto Heterogeneous Computing Systems , 1999, J. Parallel Distributed Comput..

[2]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[3]  Rajkumar Buyya,et al.  High Performance Cluster Computing: Architectures and Systems , 1999 .

[4]  Baruch Awerbuch,et al.  A cost-benefit framework for online management of a metacomputing system , 1998, ICE '98.

[5]  Michael N. Huhns,et al.  Multiagent systems and societies of agents , 1999 .

[6]  Miron Livny,et al.  Condor-a hunter of idle workstations , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[7]  Rajkumar Buyya,et al.  GridSim: a toolkit for the modeling and simulation of distributed resource management and scheduling for Grid computing , 2002, Concurr. Comput. Pract. Exp..

[8]  José A. B. Fortes,et al.  PUNCH: An architecture for Web-enabled wide-area network-computing , 2004, Cluster Computing.

[9]  Francine Berman,et al.  The AppLeS Project: A Status Report , 1997 .

[10]  Floriano Zini,et al.  Evaluation of an economy-based file replication strategy for a data grid , 2003, CCGrid 2003. 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid, 2003. Proceedings..

[11]  David Abramson,et al.  Nimrod: a tool for performing parametrised simulations using distributed workstations , 1995, Proceedings of the Fourth IEEE International Symposium on High Performance Distributed Computing.

[12]  D. Rus,et al.  Utility Driven Mobile-Agent Scheduling , 2005 .

[13]  N. Nisan,et al.  Globally distributed computation over the Internet-the POPCORN project , 1998, Proceedings. 18th International Conference on Distributed Computing Systems (Cat. No.98CB36183).

[14]  Lee W. McKnight,et al.  Pricing Internet Services: Approaches and Challenges , 2000, Computer.

[15]  David Abramson,et al.  An Economy Driven Resource Management Architecture for Global Computational Power Grids , 2000, PDPTA.

[16]  David Abramson,et al.  Economic models for resource management and scheduling in Grid computing , 2002, Concurr. Comput. Pract. Exp..

[17]  David Abramson,et al.  Nimrod/G: an architecture for a resource management and scheduling system in a global computational grid , 2000, Proceedings Fourth International Conference/Exhibition on High Performance Computing in the Asia-Pacific Region.

[18]  F. Berman,et al.  Adaptive Performance Prediction for Distributed Data-Intensive Applications , 1999, ACM/IEEE SC 1999 Conference (SC'99).

[19]  Ian T. Foster,et al.  The data grid: Towards an architecture for the distributed management and analysis of large scientific datasets , 2000, J. Netw. Comput. Appl..

[20]  Hector Garcia-Molina,et al.  Bidding for storage space in a peer-to-peer data preservation system , 2002, Proceedings 22nd International Conference on Distributed Computing Systems.

[21]  David F. Snelling,et al.  UNICORE: uniform access to supercomputing as an element of electronic commerce , 1999, Future Gener. Comput. Syst..

[22]  Nemo Semret,et al.  Auctions for Network Resource Sharing , 1997 .

[23]  Deborah Estrin,et al.  Pricing in computer networks: motivation, formulation, and example , 1993, TNET.

[24]  Gilles Fedak,et al.  XtremWeb: a generic global computing system , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[25]  Warren Smith,et al.  A Resource Management Architecture for Metacomputing Systems , 1998, JSSPP.

[26]  Lee W. McKnight,et al.  Pricing Internet Services: Proposed Improvements , 2000, Computer.

[27]  Baruch Awerbuch,et al.  An Opportunity Cost Approach for Job Assignment in a Scalable Computing Cluster , 2000, IEEE Trans. Parallel Distributed Syst..

[28]  Paul Roe,et al.  ActiveSheets: Super-Computing with Spreadsheets , 2001 .

[29]  Gerhard Weiss,et al.  Distributed Rational Decision Making , 2000 .

[30]  Rajkumar Buyya,et al.  GridBank: a Grid Accounting Services Architecture (GASA) for distributed systems sharing and integration , 2002, Proceedings International Parallel and Distributed Processing Symposium.

[31]  Ian Foster,et al.  A quality of service architecture that combines resource reservation and application adaptation , 2000, 2000 Eighth International Workshop on Quality of Service. IWQoS 2000 (Cat. No.00EX400).

[32]  David Abramson,et al.  A case for economy grid architecture for service oriented grid computing , 2001, Proceedings 15th International Parallel and Distributed Processing Symposium. IPDPS 2001.

[33]  Rajkumar Buyya,et al.  A grid service broker for scheduling distributed data-oriented applications on global grids , 2004, MGC '04.

[34]  Reid G. Smith,et al.  The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver , 1980, IEEE Transactions on Computers.

[35]  Minyi Guo,et al.  High-Performance Computing : Paradigm and Infrastructure , 2005 .

[36]  David E. Culler,et al.  Market-based cluster resource management , 2001 .

[37]  Javier Jaén Martínez,et al.  Data Management in an International Data Grid Project , 2000, GRID.

[38]  Michael Stonebraker,et al.  An economic paradigm for query processing and data migration in Mariposa , 1994, Proceedings of 3rd International Conference on Parallel and Distributed Information Systems.

[39]  Donald F. Ferguson,et al.  From Open Grid Services Infrastructure to WS-Resource Framework: Refactoring and Evolution , 2004 .

[40]  Rajkumar Buyya,et al.  Libra: a computational economy‐based job scheduling system for clusters , 2004, Softw. Pract. Exp..

[41]  Henri Casanova,et al.  Netsolve: a Network-Enabled Server for Solving Computational Science Problems , 1997, Int. J. High Perform. Comput. Appl..

[42]  Tuomas Sandholm,et al.  Distributed rational decision making , 1999 .

[43]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[44]  David E. Culler,et al.  Market-based Proportional Resource Sharing for Clusters , 2000 .

[45]  Rajkumar Buyya,et al.  Economic-based Distributed Resource Management and Scheduling for Grid Computing , 2002, ArXiv.

[46]  Giorgos Cheliotis,et al.  Architecture requirements for commercializing Grid resources , 2002, Proceedings 11th IEEE International Symposium on High Performance Distributed Computing.

[47]  Stephen Russell,et al.  Resource management in the Mungi single-address-space operating system , 1998 .

[48]  David Abramson,et al.  High performance parametric modeling with Nimrod/G: killer application for the global grid? , 2000, Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000.

[49]  Kavitha Ranganathan,et al.  Decoupling computation and data scheduling in distributed data-intensive applications , 2002, Proceedings 11th IEEE International Symposium on High Performance Distributed Computing.

[50]  David Abramson,et al.  Economic models for management of resources in peer-to-peer and grid computing , 2001, SPIE ITCom.

[51]  Rajkumar Buyya,et al.  Peer-to-Peer Grid Computing and a .NET-Based Alchemi Framework , 2006 .

[52]  Ian T. Foster,et al.  Replica selection in the Globus Data Grid , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[53]  Richard Wolski,et al.  Analyzing Market-Based Resource Allocation Strategies for the Computational Grid , 2001, Int. J. High Perform. Comput. Appl..

[54]  Francine Berman,et al.  The AppLeS Parameter Sweep Template: User-Level Middleware for the Grid , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[55]  Spyros Lalis,et al.  JaWS: An Open Market-Based Framework for Distributed Computing over the Internet , 2000, GRID.

[56]  Tad Hogg,et al.  Spawn: A Distributed Computational Economy , 1992, IEEE Trans. Software Eng..

[57]  K. Eric Drexler,et al.  Markets and computation: agoric open systems , 1988 .

[58]  John F. Karpovich,et al.  The Legion Resource Management System , 1999, JSSPP.

[59]  Richard Wolski,et al.  The network weather service: a distributed resource performance forecasting service for metacomputing , 1999, Future Gener. Comput. Syst..

[60]  Ian T. Foster,et al.  Globus: a Metacomputing Infrastructure Toolkit , 1997, Int. J. High Perform. Comput. Appl..

[61]  Erich Schikuta,et al.  Towards a cost model for distributed and replicated data stores , 2001, Proceedings Ninth Euromicro Workshop on Parallel and Distributed Processing.

[62]  Ian Pratt,et al.  Xenoservers: accountable execution of untrusted programs , 1999, Proceedings of the Seventh Workshop on Hot Topics in Operating Systems.

[63]  Rajkumar Buyya,et al.  A Market-Oriented Grid Directory Service for Publication and Discovery of Grid Service Providers and their Services , 2006, The Journal of Supercomputing.

[64]  Henri Casanova,et al.  NetSovle: A Network Server for Solving Computational Science Problems , 1996, Proceedings of the 1996 ACM/IEEE Conference on Supercomputing.

[65]  Ratul Mahajan,et al.  Controlling high bandwidth aggregates in the network , 2002, CCRV.

[66]  Rajkumar Buyya,et al.  The Gridbus toolkit for service oriented grid and utility computing: an overview and status report , 2004, 1st IEEE International Workshop on Grid Economics and Business Models, 2004. GECON 2004..

[67]  Stephen Pickles,et al.  Mini-Grids: Effective Test-Beds for GRID Application , 2000, GRID.