Using free cloud storage services for distributed evolutionary algorithms

Cloud computing, in general, is becoming part of the toolset that the scientist uses to perform compute-intensive tasks. In particular, cloud storage is an easy and convenient way of storing files that will be accessible over the Internet, but also a way of distributing those files and performing distributed computation using them. In this paper we describe how such a service commercialized by Dropbox is used for pool-based evolutionary algorithms. A prototype system is described and its peformance measured over a deceptive combinatorial optimization problem, finding that, for some type of problems and using commodity hardware, cloud storage systems can profitably be used as a platform for distributed evolutionary algorithms. Preliminary results show that Dropbox is indeed a viable alternative for execution of pool-based distributed evolutionary algorithms, showing a good scaling behavior with up to 4 computers.

[1]  Hai Jin,et al.  Cloud Types and Services , 2010, Handbook of Cloud Computing.

[2]  Juan Julián Merelo Guervós,et al.  EvAg: a scalable peer-to-peer evolutionary algorithm , 2010, Genetic Programming and Evolvable Machines.

[3]  Randy H. Katz,et al.  A view of cloud computing , 2010, CACM.

[4]  Zahir Tari,et al.  MetaCDN: Harnessing 'Storage Clouds' for high performance content delivery , 2009, J. Netw. Comput. Appl..

[5]  Erick Cantú-Paz,et al.  Topologies, Migration Rates, and Multi-Population Parallel Genetic Algorithms , 1999, GECCO.

[6]  Xavier Llorà,et al.  Meandre: Semantic-Driven Data-Intensive Flows in the Clouds , 2008, 2008 IEEE Fourth International Conference on eScience.

[7]  Alessandro Bollini,et al.  Distributed and Persistent Evolutionary Algorithms: A Design Pattern , 1999, EuroGP.

[8]  L. Darrell Whitley,et al.  Island Model genetic Algorithms and Linearly Separable Problems , 1997, Evolutionary Computing, AISB Workshop.

[9]  Eugenius Kaszkurewicz,et al.  Parallel Asynchronous Team Algorithms: Convergence and Performance Analysis , 1996, IEEE Trans. Parallel Distributed Syst..

[10]  Kalyanmoy Deb,et al.  Massive Multimodality, Deception, and Genetic Algorithms , 1992, PPSN.

[11]  Pedro S. de Souza,et al.  Genetic Algorithms in Asynchronous Teams , 1991, ICGA.

[12]  Juan Julián Merelo Guervós,et al.  Asynchronous distributed genetic algorithms with Javascript and JSON , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[13]  A. Barabasi,et al.  Parasitic computing , 2001, Nature.

[14]  Marc Sánchez Artigas,et al.  Towards the design of optimal data redundancy schemes for heterogeneous cloud storage infrastructures , 2011, Comput. Networks.

[15]  Pedro S. de Souza,et al.  Asynchronous Teams: Cooperation Schemes for Autonomous Agents , 1998, J. Heuristics.

[16]  David P. Anderson,et al.  High-performance task distribution for volunteer computing , 2005, First International Conference on e-Science and Grid Computing (e-Science'05).

[17]  Juan Julián Merelo Guervós,et al.  Testing the Intermediate Disturbance Hypothesis: Effect of Asynchronous Population Incorporation on Multi-Deme Evolutionary Algorithms , 2008, PPSN.

[18]  Yuan Zhao,et al.  A distributed pool architecture for genetic algorithms , 2009, 2009 IEEE Congress on Evolutionary Computation.

[19]  D. Ackley.  A connectionist machine for genetic hillclimbing , 1987 .

[20]  Juan Julián Merelo Guervós,et al.  Increasing GP Computing Power for Free via Desktop GRID Computing and Virtualization , 2009, 2009 17th Euromicro International Conference on Parallel, Distributed and Network-based Processing.

[21]  Rama Akkiraju,et al.  Asynchronous Teams , 2003, Handbook of Metaheuristics.

[22]  Ian T. Foster,et al.  The data grid: Towards an architecture for the distributed management and analysis of large scientific datasets , 2000, J. Netw. Comput. Appl..

[23]  Juan J. Merelo.  Fluid evolutionary algorithms , 2010, IEEE Congress on Evolutionary Computation.

[24]  Pedro A. Castillo,et al.  Increasing GP Computing Power via Volunteer Computing , 2008, ArXiv.