Grid Federation : An Economy Based , Scalable Distributed Resource Management System for Large-Scale Resource Coupling

Interest in Grid [21][22] and Peer-to-Peer (P2P) [5] computing has grown significantly over the past five years. Both are concerned with large scale resource sharing and allows a number of competitive and/or collaborative organizations to share their various resources including hardware, software, data etc. These resources range from desktops to powerful clusters of many processing units. Management of cluster resources is a key issue in grid computing while sharing and management of distributed data is of prime importance to P2P networks. Clusters of computers have emerged as mainstream parallel and distributed platforms for high-performance, high-throughput and high-availability computing. Grid [21] computing extends cluster computing idea to wide-area networks. The Grid consists of cluster resources that are usually topologically apart in multiple administrative domains, managed and owned by different organizations having different resource management policies. With the large scale growth of networks and their connectivity, it is possible to couple these cluster resources as a part of one large Grid system. Such large scale resource coupling and application management is a complex undertaking, as it introduces a number of challenges in the domain of security, resource and policy heterogeneity, resource discovery, fault tolerance, dynamic resource availability and underlying network conditions [23]. Resource sharing on Grid involves collection of resource providers (cluster owners) and resource consumers (end users) unified together towards harnessing power of distributed computational resources. Such sharing mechanisms can be master-worker based or P2P [32] where providers can be consumers as well, extending between any subset of participants. These resources and their users may even be located in different time zones. There are three key types of cluster arrangement [24], which scale from single systems to supercomputer-class compute farms that utilize thousands of processors:

[1]  Miron Livny,et al.  Condor-a hunter of idle workstations , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[2]  Tad Hogg,et al.  Spawn: A Distributed Computational Economy , 1992, IEEE Trans. Software Eng..

[3]  Michael Stonebraker,et al.  An Economic Paradigm for Query Processing and , 1994 .

[4]  Francine Berman,et al.  The AppLeS Project: A Status Report , 1997 .

[5]  Allen B. Downey,et al.  Using Queue Time Predictions for Processor Allocation , 1997, JSSPP.

[6]  Henri Casanova,et al.  Netsolve: a Network-Enabled Server for Solving Computational Science Problems , 1997, Int. J. High Perform. Comput. Appl..

[7]  Rajesh Raman,et al.  Matchmaking: distributed resource management for high throughput computing , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[8]  John F. Karpovich,et al.  The Legion Resource Management System , 1999, JSSPP.

[9]  Zhou Lei,et al.  The portable batch scheduler and the maui scheduler on linux clusters , 2000 .

[10]  Jemal H. Abawajy,et al.  Distributed hierarchical workstation cluster co-ordination scheme , 2000, Proceedings International Conference on Parallel Computing in Electrical Engineering. PARELEC 2000.

[11]  David E. Culler,et al.  REXEC: A Decentralized, Secure Remote Execution Environment for Clusters , 2000, CANPC.

[12]  David Abramson,et al.  An Economy Driven Resource Management Architecture for Global Computational Power Grids , 2000, PDPTA.

[13]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[14]  John Hebeler,et al.  Peer-to-Peer: Building Secure, Scalable, and Manageable Networks , 2001 .

[15]  Richard Wolski,et al.  G-commerce: market formulations controlling resource allocation on the computational grid , 2001, Proceedings 15th International Parallel and Distributed Processing Symposium. IPDPS 2001.

[16]  Ian Foster,et al.  On Fully Decentralized Resource Discovery in Grid Environments , 2001, GRID.

[17]  Wolfgang Gentzsch,et al.  Sun Grid Engine: towards creating a compute power grid , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[18]  Ian T. Foster,et al.  The Anatomy of the Grid: Enabling Scalable Virtual Organizations , 2001, Int. J. High Perform. Comput. Appl..

[19]  David Abramson,et al.  Economic models for resource management and scheduling in Grid computing , 2002, Concurr. Comput. Pract. Exp..

[20]  Rajkumar Buyya,et al.  Weaving computational grids: how analogous are they with electrical grids? , 2002, Comput. Sci. Eng..

[21]  Rajkumar Buyya,et al.  GridSim: a toolkit for the modeling and simulation of distributed resource management and scheduling for Grid computing , 2002, Concurr. Comput. Pract. Exp..

[22]  David P. Anderson,et al.  SETI@home: an experiment in public-resource computing , 2002, CACM.

[23]  Sanjay Jha,et al.  G-QoSM: Grid Service Discovery Using QoS Properties , 2002, Comput. Artif. Intell..

[24]  David Abramson,et al.  A Computational Economy for Grid Computing and its Implementation in the Nimrod-G Resource Brok , 2001, Future Gener. Comput. Syst..

[25]  Steven Tuecke,et al.  The Physiology of the Grid An Open Grid Services Architecture for Distributed Systems Integration , 2002 .

[26]  Artur Andrzejak,et al.  Scalable, efficient range queries for grid information services , 2002, Proceedings. Second International Conference on Peer-to-Peer Computing,.

[27]  Rajkumar Buyya,et al.  Libra: An Economy driven Job Scheduling System for Clusters , 2002, ArXiv.

[28]  Chris M. Kenyon,et al.  Grid Economics: 10 Lessons from Finance , 2003 .

[29]  A. ADoefaa,et al.  ? ? ? ? f ? ? ? ? ? , 2003 .

[30]  David R. Karger,et al.  Chord: a scalable peer-to-peer lookup protocol for internet applications , 2003, TNET.

[31]  David E. Irwin,et al.  Dynamic virtual clusters in a grid site manager , 2003, High Performance Distributed Computing, 2003. Proceedings. 12th IEEE International Symposium on.

[32]  Yuqing Zhang,et al.  Authentication and Access Control in P2P Network , 2003, GCC.

[33]  Pedro A. Szekely,et al.  MAAN: A Multi-Attribute Addressable Network for Grid Information Services , 2003, Proceedings. First Latin American Web Congress.

[34]  Rajkumar Buyya,et al.  GridBank: a Grid Accounting Services Architecture (GASA) for distributed systems sharing and integration , 2002, Proceedings International Parallel and Distributed Processing Symposium.

[35]  Paul Avery,et al.  Policy based scheduling for simple quality of service in grid computing , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[36]  José A. B. Fortes,et al.  PUNCH: An architecture for Web-enabled wide-area network-computing , 2004, Cluster Computing.

[37]  Rajkumar Buyya,et al.  Peer-to-Peer Grid Computing and a .NET-Based Alchemi Framework , 2006 .