AGOCS — Accurate Google Cloud Simulator Framework

This paper presents the Accurate Google Cloud Simulator (AGOCS) – a novel high-fidelity Cloud workload simulator based on parsing real workload traces, which can be conveniently used on a desktop machine for day-to-day research. Our simulation is based on real-world workload traces from a Google Cluster with 12.5K nodes, over a period of a calendar month. The framework is able to reveal very precise, detailed parameters of the executed jobs, tasks, nodes as well as to provide actual resource usage statistics. The system has been implemented in Scala language with focus on parallel execution, an easy-to-extend design concept. The paper presents the detailed structural framework for AGOCS, discusses our main design decisions, whilst also suggesting alternative, possibly performance enhancing future approaches. The framework is available via the Open Source GitHub repository.

[1]  Raouf Boutaba,et al.  Characterizing Task Usage Shapes in Google Compute Clusters , 2011 .

[2]  Dzmitry Kliazovich,et al.  GreenCloud: A Packet-Level Simulator of Energy-Aware Cloud Computing Data Centers , 2010, GLOBECOM.

[3]  Rajkumar Buyya,et al.  EMUSIM: an integrated emulation and simulation environment for modeling, evaluation, and validation of performance of Cloud computing applications , 2013, Softw. Pract. Exp..

[4]  Michael Abd-El-Malek,et al.  Omega: flexible, scalable schedulers for large compute clusters , 2013, EuroSys '13.

[5]  Martin Odersky,et al.  Concurrent tries with efficient non-blocking snapshots , 2012, PPoPP '12.

[6]  David A. Patterson,et al.  Rain: A Workload Generation Toolkit for Cloud Computing Applications , 2010 .

[7]  Rahul Malhotra,et al.  Study and Comparison of CloudSim Simulators in the Cloud Computing , 2013 .

[8]  Murat Kunt,et al.  ÉCOLE POLYTECHNIQUE FÉDÉRALE DE LAUSANNE , 2000 .

[9]  Archana Ganapathi,et al.  Statistics-driven workload modeling for the Cloud , 2010, 2010 IEEE 26th International Conference on Data Engineering Workshops (ICDEW 2010).

[10]  Guanying Wang,et al.  Towards Synthesizing Realistic Workload Traces for Studying the Hadoop Ecosystem , 2011, 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems.

[11]  Chita R. Das,et al.  Towards characterizing cloud backend workloads: insights from Google compute clusters , 2010, PERV.

[12]  Vladimir Getov,et al.  A Meta-Heuristic Load Balancer for Cloud Computing Systems , 2015, 2015 IEEE 39th Annual Computer Software and Applications Conference.

[13]  Alan Mycroft,et al.  Java 8 in Action: Lambdas, Streams, and Functional-Style Programming , 2014 .

[14]  Jie Xu,et al.  An Approach for Characterizing Workloads in Google Cloud to Derive Realistic Resource Utilization Models , 2013, 2013 IEEE Seventh International Symposium on Service-Oriented System Engineering.

[15]  Rajkumar Buyya,et al.  NetworkCloudSim: Modelling Parallel Applications in Cloud Simulations , 2011, 2011 Fourth IEEE International Conference on Utility and Cloud Computing.

[16]  Dzmitry Kliazovich,et al.  GreenCloud: a packet-level simulator of energy-aware cloud computing data centers , 2010, The Journal of Supercomputing.

[17]  Chita R. Das,et al.  Modeling and synthesizing task placement constraints in Google compute clusters , 2011, SoCC.

[18]  Alexandru Iosup,et al.  The Grid Workloads Archive , 2008, Future Gener. Comput. Syst..