Autonomic load balancing mechanisms in the P2P desktop grid

Peer-to-Peer (P2P) desktop grid computing systems circumvent the performance bottleneck and limited scalability of centralized Grid architectures resulting in a massively scalable and robust system. We have designed a set of protocols that implement a distributed, decentralized desktop grid via P2P techniques. Incoming jobs having different types of resource requirements are matched with system nodes through proximity in an N-dimensional resource space. In this paper, we address problems that arise from static load balancing mechanisms for assigning jobs to nodes that can arise for various reasons, including the heterogeneity of the available nodes or the jobs to be run, and from stale information in the P2P system. We greatly improve upon our prior work by providing lightweight yet effective dynamic load balancing mechanisms to overcome load imbalances caused by the limitations of the initial static job assignment scheme. Unlike other systems, we can effectively support resource constraints of jobs during the course of redistribution since we simplify the problem of matchmaking through building a multi-dimensional resource space and mapping jobs and nodes to this space. Throughout extensive simulation results, we show that dynamic load balancing makes the overall system more scalable, by improving system throughput and response time with low additional overhead.

[1]  Hanan Samet,et al.  Foundations of multidimensional and metric data structures , 2006, Morgan Kaufmann series in data management systems.

[2]  Yong Zhao,et al.  Many-task computing for grids and supercomputers , 2008, 2008 Workshop on Many-Task Computing on Grids and Supercomputers.

[3]  Douglas Thain,et al.  Distributed computing in practice: the Condor experience , 2005, Concurr. Pract. Exp..

[4]  Denis Caromel,et al.  Peer-to-peer for computational grids: mixing clusters and desktop machines , 2007, Parallel Comput..

[5]  David P. Anderson,et al.  SETI@home: an experiment in public-resource computing , 2002, CACM.

[6]  Richard M. Karp,et al.  Load balancing in dynamic structured P2P systems , 2004, IEEE INFOCOM 2004.

[7]  Indranil Gupta,et al.  Peer-to-peer discovery of computational resources for Grid applications , 2005, The 6th IEEE/ACM International Workshop on Grid Computing, 2005..

[8]  Alexandru Iosup,et al.  How are Real Grids Used? The Analysis of Four Grid Traces and Its Implications , 2006, 2006 7th IEEE/ACM International Conference on Grid Computing.

[9]  Zhao Zhang,et al.  Towards Loo on , 2008 .

[10]  David P. Anderson,et al.  BOINC: a system for public-resource computing and storage , 2004, Fifth IEEE/ACM International Workshop on Grid Computing.

[11]  Edward Walker,et al.  Creating personal adaptive clusters for managing scientific jobs in a distributed computing environment , 2006, 2006 IEEE Challenges of Large Applications in Distributed Environments.

[12]  Miron Livny,et al.  Condor and preemptive resume scheduling , 2004 .

[13]  Alexandru Iosup,et al.  Grid Computing Workloads , 2011, IEEE Internet Computing.

[14]  Amin Vahdat,et al.  Design and implementation tradeoffs for wide-area resource discovery , 2005, HPDC-14. Proceedings. 14th IEEE International Symposium on High Performance Distributed Computing, 2005..

[15]  Domenico Talia,et al.  Peer-to-Peer resource discovery in Grids: Models and systems , 2007, Future Gener. Comput. Syst..

[16]  Suresh Jagannathan,et al.  Unstructured peer-to-peer networks for sharing processor cycles , 2006, Parallel Comput..

[17]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[18]  P. Oscar Boykin,et al.  Balanced Overlay Networks (BON): An Overlay Technology for Decentralized Load Balancing , 2004, IEEE Transactions on Parallel and Distributed Systems.

[19]  Yiming Hu,et al.  Efficient, proximity-aware load balancing for DHT-based P2P systems , 2005, IEEE Transactions on Parallel and Distributed Systems.

[20]  Mukesh Singhal,et al.  Load distributing for locally distributed systems , 1992, Computer.

[21]  Yong Zhao,et al.  Falkon: a Fast and Light-weight tasK executiON framework , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[22]  Patrick Crowley,et al.  Dynamic thread assignment on heterogeneous multiprocessor architectures , 2006, CF '06.

[23]  Domenico Talia,et al.  A Super-Peer Model for Building Resource Discovery Services in Grids: Design and Simulation Analysis , 2005, EGC.

[24]  Miron Livny,et al.  Mechanisms for High Throughput Computing , 1997 .

[25]  Margo I. Seltzer,et al.  Scooped, Again , 2003, IPTPS.

[26]  Bobby Bhattacharjee,et al.  Using content-addressable networks for load balancing in desktop grids , 2007, HPDC '07.

[27]  原田 秀逸.  私の computer 環境 , 1998 .

[28]  Alexandru Iosup,et al.  Grid Computing Workloads : Bags of Tasks , Workflows , Pilots , and Others , 2010 .

[29]  Virginia Mary Lo,et al.  WaveGrid: a scalable fast-turnaround heterogeneous peer-based desktop grid system , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[30]  B. Allen,et al.  Designing a Runtime System for Volunteer Computing , 2006, ACM/IEEE SC 2006 Conference (SC'06).

[31]  Arun K. Somani,et al.  CompuP2P: An Architecture for Internet Computing Using Peer-to-Peer Networks , 2006, IEEE Transactions on Parallel and Distributed Systems.

[32]  Bobby Bhattacharjee,et al.  Trade-offs in matching jobs and balancing load for distributed desktop grids , 2008, Future Gener. Comput. Syst..

[33]  Bobby Bhattacharjee,et al.  Integrating categorical resource types into a P2P desktop grid system , 2008, 2008 9th IEEE/ACM International Conference on Grid Computing.