Bridging Concepts and Practice in eScience via Simulation-Driven Engineering

The CyberInfrastructure (CI) has been the object of intensive research and development in the last decade, resulting in a rich set of abstractions and interoperable software implementations that are used in production today for supporting ongoing and breakthrough scientific discoveries. A key challenge is the development of tools and application execution frameworks that are robust in current and emerging CI configurations, and that can anticipate the needs of upcoming CI applications. This paper presents WRENCH, a framework that enables simulation-driven engineering for evaluating and developing CI application execution frameworks. WRENCH provides a set of high-level simulation abstractions that serve as building blocks for developing custom simulators. These abstractions rely on the scalable and accurate simulation models that are provided by the SimGrid simulation framework. Consequently, WRENCH makes it possible to build, with minimum software development effort, simulators that that can accurately and scalably simulate a wide spectrum of large and complex CI scenarios. These simulators can then be used to evaluate and/or compare alternate platform, system, and algorithm designs, so as to drive the development of CI solutions for current and emerging applications.

[1]  André Schiper,et al.  Theory and Practice in Distributed Systems , 1995, Lecture Notes in Computer Science.

[2]  Henri Casanova,et al.  Versatile, scalable, and accurate simulation of distributed applications and platforms , 2014, J. Parallel Distributed Comput..

[3]  Miron Livny,et al.  Pegasus, a workflow management system for science automation , 2015, Future Gener. Comput. Syst..

[4]  Henri Casanova,et al.  WRENCH: A Framework for Simulating Workflow Management Systems , 2018, 2018 IEEE/ACM Workflows in Support of Large-Scale Science (WORKS).

[5]  Douglas Thain,et al.  Harnessing parallelism in multicore clusters with the All-Pairs, Wavefront, and Makeflow abstractions , 2010, Cluster Computing.

[6]  Hugues Benoit-Cattin,et al.  Modeling Distributed Platforms from Application Traces for Realistic File Transfer Simulation , 2017, 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID).

[7]  Henri Casanova,et al.  Teaching Parallel and Distributed Computing Concepts in Simulation with WRENCH , 2019, 2019 IEEE/ACM Workshop on Education for High-Performance Computing (EduHPC).

[8]  Henri Casanova,et al.  On the validity of flow-level tcp network models for grid and cloud simulations , 2013, TOMC.

[9]  Henri Casanova,et al.  Accurately Simulating Energy Consumption of I/O-Intensive Scientific Workflows , 2019, ICCS.

[10]  Todd R. Andel,et al.  On the credibility of manet simulations , 2006, Computer.