Automatic grid assembly by promoting collaboration in peer-to-peer grids

Currently, most computational grids (systems allowing transparent sharing of computing resources across organizational boundaries) are assembled using human negotiation. This procedure does not scale well, and is too inflexible to allow for large open grids. Peer-to-peer (P2P) grids present an alternative way to build grids with many sites. However, to actually assemble a large grid, peers must have an incentive to provide resources to the system. In this paper we present an incentive mechanism called the Network of Favors, which makes it in the interest of each participating peer to contribute its spare resources. We show through simulations with up to 10,000 peers and experiments with software implementing the mechanism in a deployed system that the Network of Favors promotes collaboration in a simple, robust and scalable fashion. We also discuss experiences of using OurGrid, a grid based on this mechanism.

[1]  Nazareno Andrade,et al.  Discouraging free riding in a peer-to-peer CPU-sharing grid , 2004, Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004..

[2]  Francisco Vilar Brasileiro,et al.  Exploiting Replication and Data Reuse to Efficiently Schedule Data-Intensive Applications on Grids , 2004, JSSPP.

[3]  Nazareno Andrade,et al.  Accurate autonomous accounting in peer-to-peer Grids , 2005, MGC '05.

[4]  Sujata Banerjee,et al.  NodeWiz: peer-to-peer resource discovery for grids , 2005, CCGrid 2005. IEEE International Symposium on Cluster Computing and the Grid, 2005..

[5]  Santosh K. Shrivastava,et al.  A System for Fault-Tolerance Execution of Data and Compute Intensive Programs over a Network of Workstations , 1996, Euro-Par, Vol. I.

[6]  Ian J. Taylor,et al.  Distributed computing with Triana on the Grid , 2005, Concurr. Pract. Exp..

[7]  Gilles Fedak,et al.  XtremWeb: a generic global computing system , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[8]  Hector Garcia-Molina,et al.  The Eigentrust algorithm for reputation management in P2P networks , 2003, WWW '03.

[9]  B. Cohen,et al.  Incentives Build Robustness in Bit-Torrent , 2003 .

[10]  Luis F. G. Sarmenta.  Sabotage-tolerance mechanisms for volunteer computing systems , 2002, Future Gener. Comput. Syst..

[11]  Ernesto Damiani,et al.  Managing and Sharing Servents' Reputations in P2P Systems , 2003, IEEE Trans. Knowl. Data Eng..

[12]  Ion Stoica,et al.  Incentives for Cooperation in Peer-to-Peer Networks , 2003 .

[13]  Christian Grothoff,et al.  An Excess-Based Economic Model for Resource Allocation in Peer-to-Peer Networks , 2005 .

[14]  Hans F. Hoffmann.  The Lhc Computing Grid , 2004 .

[15]  Geoff Coulson,et al.  Free Riding on Gnutella Revisited: The Bell Tolls? , 2005, IEEE Distributed Syst. Online.

[16]  Francisco Vilar Brasileiro,et al.  Trading Cycles for Information: Using Replication to Schedule Bag-of-Tasks Applications on Computational Grids , 2003, Euro-Par.

[17]  Miron Livny,et al.  Condor-a hunter of idle workstations , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[18]  Bernardo A. Huberman,et al.  Tycoon: A Market-Based Resource Allocation System , 2004, ArXiv.

[19]  Ian T. Foster,et al.  On Death, Taxes, and the Convergence of Peer-to-Peer and Grid Computing , 2003, IPTPS.

[20]  R. Gardner.  Grid3 : An Application Grid Laboratory for Science , 2005 .

[21]  Ion Stoica,et al.  Robust incentive techniques for peer-to-peer networks , 2004, EC '04.

[22]  Nazareno Andrade,et al.  Labs of the World, Unite!!! , 2006, Journal of Grid Computing.

[23]  W. Bean.  On death. , 1958, A.M.A. archives of internal medicine.

[24]  Amin Vahdat,et al.  Bootstrapping a Distributed Computational Economy with Peer-to-Peer Bartering , 2003 .

[25]  Andrew Warfield,et al.  Xen and the art of virtualization , 2003, SOSP '03.

[26]  Ian Goldberg,et al.  A Secure Environment for Untrusted Helper Applications ( Confining the Wily Hacker ) , 1996 .

[27]  Daniel Zappala,et al.  Cluster Computing on the Fly : P 2 P Scheduling of Idle Cycles in the Internet , 2004 .

[28]  Emin Gün Sirer,et al.  KARMA : A Secure Economic Framework for Peer-to-Peer Resource Sharing , 2003 .

[29]  Amin Vahdat,et al.  SHARP: an architecture for secure resource peering , 2003, SOSP '03.

[30]  John R. Douceur,et al.  The Sybil Attack , 2002, IPTPS.

[31]  Francisco Brasileiro,et al.  Grid Computing for Bag of Tasks Applications , 2003 .

[32]  Sujata Banerjee,et al.  A large scale fault-tolerant grid information service , 2006, MCG '06.

[33]  Christian S. Collberg,et al.  Watermarking, Tamper-Proofing, and Obfuscation-Tools for Software Protection , 2002, IEEE Trans. Software Eng..

[34]  Ian J. Taylor,et al.  Distributed P2P computing within Triana: a galaxy visualization test case , 2003, Proceedings International Parallel and Distributed Processing Symposium.

[35]  Ian T. Foster,et al.  The Anatomy of the Grid: Enabling Scalable Virtual Organizations , 2001, Int. J. High Perform. Comput. Appl..

[36]  Beng Chin Ooi,et al.  Managing Trust in Peer-to-Peer Systems Using Reputation-Based Techniques , 2003, WAIM.

[37]  Rajkumar Buyya,et al.  Compute Power Market: towards a market-oriented grid , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[39]  T. Yamagishi,et al.  Improving the Lemons Market with a Reputation System : An Experimental Study of Internet Auctioning , 2002 .

[40]  Hector Garcia-Molina,et al.  EigenRep: Reputation Management in P2P Networks , 2003 .

[41]  Fernando M. A. Silva,et al.  P3: Parallel Peer to Peer An Internet Parallel Programming Environment , 2002 .

[42]  Thomas Hérault,et al.  Computing on large-scale distributed systems: XtremWeb architecture, programming models, security, tests and convergence with grid , 2005, Future Gener. Comput. Syst..

[43]  Stefan Saroiu,et al.  A Measurement Study of Peer-to-Peer File Sharing Systems , 2001 .

[44]  Hai Jin.  ChinaGrid: Making Grid Computing a Reality , 2004, ICADL.

[45]  David K. Y. Yau,et al.  An incentive mechanism for P2P networks , 2004, 24th International Conference on Distributed Computing Systems, 2004. Proceedings..

[46]  David Abramson,et al.  A Computational Economy for Grid Computing and its Implementation in the Nimrod-G Resource Brok , 2001, Future Gener. Comput. Syst..

[47]  Andrew A. Chien,et al.  Resource Management for Rapid Application Turnaround on Enterprise Desktop Grids , 2004, Proceedings of the ACM/IEEE SC2004 Conference.

[48]  Eytan Adar,et al.  Free Riding on Gnutella , 2000, First Monday.

[49]  David K. Y. Yau,et al.  Incentive and Service Differentiation in P2P Networks: A Game Theoretic Approach , 2006, IEEE/ACM Transactions on Networking.

[50]  Ian T. Foster,et al.  Mapping the Gnutella Network: Macroscopic Properties of Large-Scale Peer-to-Peer Systems , 2002, IPTPS.

[51]  Nazareno Andrade,et al.  When can an autonomous reputation scheme discourage free-riding in a peer-to-peer system? , 2004, IEEE International Symposium on Cluster Computing and the Grid, 2004. CCGrid 2004..

[52]  Richard Wolski,et al.  Analyzing Market-Based Resource Allocation Strategies for the Computational Grid , 2001, Int. J. High Perform. Comput. Appl..

[53]  David K. Y. Yau,et al.  A game theoretic approach to provide incentive and service differentiation in P2P networks , 2004, SIGMETRICS '04/Performance '04.

[54]  Paul Resnick,et al.  Reputation systems , 2000, CACM.

[55]  Nazareno Andrade,et al.  OurGrid: An Approach to Easily Assemble Grids with Equitable Resource Sharing , 2003, JSSPP.

[56]  Y. Charlie Hu,et al.  A Self-Organizing Flock of Condors , 2003, ACM/IEEE SC 2003 Conference (SC'03).

[57]  Cynthia Bailey Lee,et al.  Are User Runtime Estimates Inherently Inaccurate? , 2004, JSSPP.

[58]  Amin Vahdat,et al.  Why Markets Could (But Don't Currently) Solve Resource Allocation Problems in Systems , 2005, HotOS.