Distributed simulation optimization and parameter exploration framework for the cloud

Abstract Simulation models are becoming an increasingly popular tool for the analysis and optimization of complex real systems in different fields. Finding an optimal system design requires performing a large sweep over the parameter space in an organized way. Hence, the model optimization process is extremely demanding from a computational point of view, as it requires careful, time-consuming, complex orchestration of coordinated executions. In this paper, we present the design of SOF (Simulation Optimization and exploration Framework in the cloud), a framework which exploits the computing power of a cloud computational environment in order to carry out effective and efficient simulation optimization strategies. SOF offers several attractive features. Firstly, SOF requires “zero configuration”, as it does not require any additional software installed on the remote node; only standard Apache Hadoop and SSH access are sufficient. Secondly, SOF is transparent to the user, since the user is totally unaware that the system operates on a distributed environment. Finally, SOF is highly customizable and programmable, since it enables the running of different simulation optimization scenarios using diverse programming languages – provided that the hosting platform supports them – and different simulation toolkits, as developed by the modeler. The tool has been fully developed and is available on a public repository 1 under the terms of the open source Apache License. It has been tested and validated on several private platforms, such as a dedicated cluster of workstations, as well as on public platforms, including the Hortonworks Data Platform and Amazon Web Services Elastic MapReduce solution.

[1]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[2]  Enrique Alba,et al.  Parallel metaheuristics: recent advances and new trends , 2012, Int. Trans. Oper. Res..

[3]  Susan R. Hunter,et al.  Comparing Message Passing Interface and MapReduce for large-scale parallel ranking and selection , 2015, 2015 Winter Simulation Conference (WSC).

[4]  Carmine Spagnuolo,et al.  From desktop to Large-Scale Model Exploration with Swift/T , 2016, 2016 Winter Simulation Conference (WSC).

[5]  George Bosilca,et al.  Open MPI: Goals, Concept, and Design of a Next Generation MPI Implementation , 2004, PVM/MPI.

[6]  R. H. Myers,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[7]  Arnold L. Rosenberg,et al.  Optimal sharing of bags of tasks in heterogeneous clusters , 2003, SPAA '03.

[8]  Guillaume Hutzler,et al.  Parameter Space Exploration of Agent-Based Models , 2005, KES.

[9]  Sebastien Rey-Coyrehourcq,et al.  OpenMOLE, a workflow engine specifically tailored for the distributed exploration of simulation models , 2013, Future Gener. Comput. Syst..

[10]  U. Netlogo Wilensky,et al.  Center for Connected Learning and Computer-Based Modeling , 1999 .

[11]  Sean Luke,et al.  MASON: A Multiagent Simulation Environment , 2005, Simul..

[12]  Warren B. Powell,et al.  A Knowledge-Gradient Policy for Sequential Information Collection , 2008, SIAM J. Control. Optim..

[13]  Michael J. North,et al.  Complex adaptive systems modeling with Repast Simphony , 2013, Complex Adapt. Syst. Model..

[14]  Susan R. Hunter,et al.  A comparison of two parallel ranking and selection procedures , 2014, Proceedings of the Winter Simulation Conference 2014.

[15]  Matthias Ehrgott,et al.  Multiple criteria decision analysis: state of the art surveys , 2005 .

[16]  Forrest Stonedahl,et al.  Genetic algorithms for the exploration of parameter spaces in agent-based models , 2011 .

[17]  Zhao Zhang,et al.  Parallel Scripting for Applications at the Petascale and Beyond , 2009, Computer.

[18]  Telmo da Silva Morais.  Survey on Frameworks for Distributed Computing: Hadoop, Spark and Storm , 2015 .

[19]  R. Ackoff.  Towards a System of Systems Concepts , 1971 .

[20]  Michael C. Fu,et al.  Handbook of Simulation Optimization , 2014 .

[21]  Howard Gobioff,et al.  The Google file system , 2003, SOSP '03.

[22]  Robert Klein,et al.  Simulation-based optimization of an agent-based simulation , 2014 .

[23]  Vladimir L. Kharitonov,et al.  Distributed simulation of hybrid systems with AnyLogic and HLA , 2002, Future Gener. Comput. Syst..

[24]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[25]  Russell R. Barton,et al.  Simulation metamodels , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).

[26]  Enver Yücesan,et al.  Guest editors' introduction to special issue on the first INFORMS simulation society research workshop , 2010, TOMC.

[27]  Sanjay Ghemawat,et al.  MapReduce: simplified data processing on large clusters , 2008, CACM.

[28]  Barry L. Nelson,et al.  Optimization via simulation over discrete decision variables , 2010 .

[29]  David Ginsbourger,et al.  Expected Improvements for the Asynchronous Parallel Global Optimization of Expensive Functions: Potentials and Challenges , 2012, LION.

[30]  André I. Khuri,et al.  Response surface methodology: 1966–1988 , 1989 .

[31]  Joshua M. Epstein,et al.  Growing Artificial Societies: Social Science from the Bottom Up , 1996 .

[32]  Michael J. North,et al.  A Declarative Model Assembly Infrastructure for Verification and Validation , 2006, WCSS.

[33]  Loo Hay Lee,et al.  Stochastic Simulation Optimization - An Optimal Computing Budget Allocation , 2010, System Engineering and Operations Research.

[34]  Hairong Kuang,et al.  The Hadoop Distributed File System , 2010, 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST).

[35]  André I. Khuri,et al.  Response surface methodology , 2010 .

[36]  Julie L. Swann,et al.  Simple Procedures for Selecting the Best Simulated System When the Number of Alternatives is Large , 2001, Oper. Res..

[37]  Chun-Hung Chen,et al.  A Review of Optimal Computing Budget Allocation Algorithms for Simulation Optimization Problem , 2010 .

[38]  Leandro Nunes de Castro,et al.  Fundamentals of Natural Computing - Basic Concepts, Algorithms, and Applications , 2006, Chapman and Hall / CRC computer and information science series.

[39]  Chun-Hung Chen,et al.  Opportunity Cost and OCBA Selection Procedures in Ordinal Optimization for a Fixed Number of Alternative Systems , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[40]  Charles M. Macal,et al.  Tutorial on agent-based modeling and simulation , 2005 .

[41]  Andreas Krause,et al.  Parallelizing Exploration-Exploitation Tradeoffs with Gaussian Process Bandit Optimization , 2012, ICML.

[42]  Stephen E. Chick,et al.  New Two-Stage and Sequential Procedures for Selecting the Best Simulated System , 2001, Oper. Res..

[43]  Steven L. Lytinen,et al.  Agent-based Simulation Platforms: Review and Development Recommendations , 2006, Simul..

[44]  Ihsan Sabuncuoglu,et al.  Simulation optimization: A comprehensive review on theory and applications , 2004 .

[45]  Loo Hay Lee,et al.  Simulation optimization using the cross-entropy method with optimal computing budget allocation , 2010, TOMC.

[46]  Bogumił Kamiński,et al.  On parallel policies for ranking and selection problems , 2018 .

[47]  Jürgen Branke,et al.  Selecting a Selection Procedure , 2007, Manag. Sci..

[48]  Fred W. Glover,et al.  Simulation optimization: a review, new developments, and applications , 2005, Proceedings of the Winter Simulation Conference, 2005..

[49]  Barry L. Nelson,et al.  Stochastic kriging for simulation metamodeling , 2008, WSC 2008.

[50]  Sebastián Lozano,et al.  Metaheuristic optimization frameworks: a survey and benchmarking , 2011, Soft Computing.