Fast and scalable simulation of volunteer computing systems using SimGrid

Advances in internetworking technology and the decreasing cost-performance ratio of commodity computing components have enabled Volunteer Computing (VC). VC platforms aggregate tens or hundreds of thousands of hosts. These hosts are typically volatile, which raises difficult research questions. Most research in this area relies on simulation. The main issue when developing VC simulators is scalability: How to perform simulations of large-scale VC platforms with reasonable amounts of memory and reasonably fast? To achieve scalability, state-of-the-art VC simulators employ simplistic simulation models and/or target on narrow platform and application scenarios. In this paper we enable VC simulations using the general-purpose SimGrid simulation framework, which provides significantly more realistic and flexible simulation capabilities than the aforementioned simulators. Our key contribution is a set of improvements to SimGrid so that it brings these benefits to VC simulations while achieving good scalability.

[1]  Henri Casanova,et al.  Simgrid: a toolkit for the simulation of application scheduling , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[2]  David P. Anderson,et al.  Performance Evaluation of Scheduling Policies for Volunteer Computing , 2007, Third IEEE International Conference on e-Science and Grid Computing (e-Science 2007).

[3]  Rajkumar Buyya,et al.  GridSim: a toolkit for the modeling and simulation of distributed resource management and scheduling for Grid computing , 2002, Concurr. Comput. Pract. Exp..

[4]  Andrew A. Chien,et al.  Resource Management for Rapid Application Turnaround on Enterprise Desktop Grids , 2004, Proceedings of the ACM/IEEE SC2004 Conference.

[5]  Jan Broeckhove,et al.  Scalability of Grid Simulators: An Evaluation , 2008, Euro-Par.

[6]  Harold Enrique Castro Barrera,et al.  Desktop Grids and Volunteer Computing Systems , 2012 .

[7]  Nazareno Andrade,et al.  Labs of the World, Unite!!! , 2006, Journal of Grid Computing.

[8]  Richard Wolski,et al.  Modeling Machine Availability in Enterprise and Wide-Area Distributed Computing Environments , 2005, Euro-Par.

[9]  David P. Anderson,et al.  BOINC: a system for public-resource computing and storage , 2004, Fifth IEEE/ACM International Workshop on Grid Computing.

[10]  Gilles Fedak,et al.  Desktop Grids: From Volunteer Distributed Computing to High Throughput Computing Production Platforms , 2010 .

[11]  Hermes Senger,et al.  Improving scalability of Bag-of-Tasks applications running on master-slave platforms , 2009, Parallel Comput..

[12]  David P. Anderson,et al.  SETI@home: an experiment in public-resource computing , 2002, CACM.

[13]  George F. Riley,et al.  The Georgia Tech Network Simulator , 2003, MoMeTools '03.

[14]  Thomas Hérault,et al.  Computing on large-scale distributed systems: XtremWeb architecture, programming models, security, tests and convergence with grid , 2005, Future Gener. Comput. Syst..

[15]  Alexandru Iosup,et al.  The Failure Trace Archive: Enabling Comparative Analysis of Failures in Diverse Distributed Systems , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[16]  Dimitri P. Bertsekas,et al.  Data Networks , 1986 .

[17]  Kenichi Hagihara,et al.  Computing Low Latency Batches with Unreliable Workers in Volunteer Computing Environments , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[18]  Henri Casanova,et al.  SimGrid: A Generic Framework for Large-Scale Distributed Experiments , 2008, Tenth International Conference on Computer Modeling and Simulation (uksim 2008).

[19]  Steven H. Low,et al.  A duality model of TCP and queue management algorithms , 2003, TNET.

[20]  Rakesh Kumar,et al.  Stochastic Fluid Theory for P2P Streaming Systems , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[21]  Francisco Vilar Brasileiro,et al.  Exploiting Replication and Data Reuse to Efficiently Schedule Data-Intensive Applications on Grids , 2004, JSSPP.

[22]  Arnaud Legrand,et al.  Accuracy study and improvement of network simulation in the SimGrid framework , 2009, SimuTools.

[23]  David P. Anderson,et al.  The Effectiveness of Threshold-Based Scheduling Policies in BOINC Projects , 2006, 2006 Second IEEE International Conference on e-Science and Grid Computing (e-Science'06).

[24]  R. Srikant,et al.  Modeling and performance analysis of BitTorrent-like peer-to-peer networks , 2004, SIGCOMM '04.

[25]  Ian Wakeman,et al.  Towards Yet Another Peer-to-Peer Simulator , 2006 .

[26]  Gilles Fedak,et al.  Characterizing resource availability in enterprise desktop grids , 2007, Future Gener. Comput. Syst..

[27]  David P. Anderson,et al.  Local Scheduling for Volunteer Computing , 2007, 2007 IEEE International Parallel and Distributed Processing Symposium.

[28]  David P. Anderson,et al.  EmBOINC: An emulator for performance analysis of BOINC projects , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[29]  Patricia J. Teller,et al.  SimBA: A Discrete Event Simulator for Performance Prediction of Volunteer Computing Projects , 2007, 21st International Workshop on Principles of Advanced and Distributed Simulation (PADS'07).

[30]  Andrew A. Chien,et al.  Henri Casanova , 2022 .

[31]  Nazareno Andrade,et al.  A Reciprocation-Based Economy for Multiple Services in Peer-to-Peer Grids , 2006, Sixth IEEE International Conference on Peer-to-Peer Computing (P2P'06).