Configurable performance analysis and evaluation framework for cloud systems

Performance analysis and evaluation are considered as key infrastructures for distributed computing platforms, such as clusters and grids. Recently, cloud computing has become an emerging commercial infrastructure paradigm, which promises to eliminate the need for maintaining expensive computing facilities by companies and institutes. However, few efforts are taken to address the issue of performance evaluation in cloud environments. In this paper, we present a configurable performance evaluation analysis and evaluation framework, which is aiming to provide users and researchers an easy-to-use toolkit to evaluate their cloud system's runtime performance, or compare the performance when different resource management policy and task scheduling algorithms are taken into account. Extensive experiments are conducted to investigate the effectiveness of the system implementation, and the results indicate that its configurable feature is of significantly usefulness when conducting performance comparing under different scenarios.

[1]  Albert Y. Zomaya,et al.  Profit-Driven Service Request Scheduling in Clouds , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[2]  Dejan S. Milojicic,et al.  Eucalyptus: Delivering a Private Cloud , 2011, Computer.

[3]  Emmanuel Jeannot,et al.  On the distribution of sequential jobs in random brokering for heterogeneous computational grids , 2006, IEEE Transactions on Parallel and Distributed Systems.

[4]  Dick H. J. Epema,et al.  Experiences with the KOALA co-allocating scheduler in multiclusters , 2005, CCGrid 2005. IEEE International Symposium on Cluster Computing and the Grid, 2005..

[5]  Henri Casanova,et al.  Benefits and Drawbacks of Redundant Batch Requests , 2007, Journal of Grid Computing.

[6]  Matei Ripeanu,et al.  Amazon S3 for science grids: a viable solution? , 2008, DADC '08.

[7]  Ewa Deelman,et al.  The cost of doing science on the cloud: the Montage example , 2008, HiPC 2008.

[8]  Roger Clarke,et al.  User Requirements for Cloud Computing Architecture , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[9]  Alexandru Iosup,et al.  C-Meter: A Framework for Performance Analysis of Computing Clouds , 2009, 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid.

[10]  Jianxin Li,et al.  An Efficient Resource Management System for On-Line Virtual Cluster Provision , 2009, 2009 IEEE International Conference on Cloud Computing.

[11]  N. Nagaveni,et al.  Design and Implementation of an Efficient Two-Level Scheduler for Cloud Computing Environment , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[12]  Richard Wolski,et al.  The Eucalyptus Open-Source Cloud-Computing System , 2009, 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid.

[13]  Alexandru Iosup,et al.  GRENCHMARK: A Framework for Analyzing, Testing, and Comparing Grids , 2006, Sixth IEEE International Symposium on Cluster Computing and the Grid (CCGRID'06).

[14]  Franck Cappello,et al.  Scalability Comparison of Four Host Virtualization Tools , 2007, Journal of Grid Computing.

[15]  Victor I. Chang,et al.  A Categorisation of Cloud Computing Business Models , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[16]  Alexandru Iosup,et al.  Performance Analysis of Cloud Computing Services for Many-Tasks Scientific Computing , 2011, IEEE Transactions on Parallel and Distributed Systems.

[17]  Christian Engelmann,et al.  Proactive fault tolerance for HPC with Xen virtualization , 2007, ICS '07.

[18]  Uwe Schwiegelshohn,et al.  Theory and Practice in Parallel Job Scheduling , 1997, JSSPP.

[19]  Howard Gobioff,et al.  The Google file system , 2003, SOSP '03.

[20]  Hung-Yu Wei,et al.  Dynamic Auction Mechanism for Cloud Resource Allocation , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[21]  Ian T. Foster,et al.  The Design, Usage, and Performance of GRUBER: A Grid Usage Service Level Agreement based BrokERing Infrastructure , 2006, Journal of Grid Computing.

[22]  Andrew Warfield,et al.  Xen and the art of virtualization , 2003, SOSP '03.

[23]  Ashraf Aboulnaga,et al.  Database systems on virtual machines: How much do you lose? , 2008, 2008 IEEE 24th International Conference on Data Engineering Workshop.

[24]  Willy Zwaenepoel,et al.  Diagnosing performance overheads in the xen virtual machine environment , 2005, VEE '05.

[25]  Dan Tsafrir,et al.  Modeling User Runtime Estimates , 2005, JSSPP.

[26]  Yu Deng,et al.  Evolution of the IBM Cloud: Enabling an enterprise cloud services ecosystem , 2011, IBM J. Res. Dev..

[27]  Francine Berman,et al.  A comprehensive model of the supercomputer workload , 2001 .

[28]  Dror G. Feitelson,et al.  The workload on parallel supercomputers: modeling the characteristics of rigid jobs , 2003, J. Parallel Distributed Comput..