CROWNBench: A Grid Performance Testing System Using Customizable Synthetic Workload

The Grid middleware must be developed iteratively and incrementally, so Grid performance testing is critical for middleware developers of Grid system. Considering the special characters of Grid system, in order to gain meaningful and comprehensive results of performance testing, it is necessary to implement testing on real Grid environment with various types of workload. CROWNBench, as described in this paper, is a system for helping Grid middleware developers to evaluate middleware design and implement using customizable synthetic workload. Middleware developers can customize testing workload basing on the model of Grid workload derived from real workload traces, including its structure and parameters, and then workload is synthesized automatically and contained jobs will be submitted by CROWNBench in a distributed manner. CROWNBench defines several metrics for measuring Grid performance as automatic testing results. The experiment, which used CROWNBench to test the performance of Grid system with CROWN Grid middleware, shows that the system already finished have accomplished its prospective goal. It can implement Grid performance testing in an efficient, flexible, controllable, replayable and automatic way to help middleware developers evaluate and improve their products effectively.

[1]  Dror G. Feitelson,et al.  The workload on parallel supercomputers: modeling the characteristics of rigid jobs , 2003, J. Parallel Distributed Comput..

[2]  Giuseppe Serazzi,et al.  Workload characterization: a survey , 1993, Proc. IEEE.

[3]  Marios D. Dikaiakos,et al.  GridBench: a tool for benchmarking grids , 2003, Proceedings. First Latin American Web Congress.

[4]  Michael A. Frumkin,et al.  NAS Grid Benchmarks: A Tool for Grid Space Exploration , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.

[5]  Hui Li.  Performance Evaluation in Grid Computing: A Modeling and Prediction Perspective , 2007, Seventh IEEE International Symposium on Cluster Computing and the Grid (CCGrid '07).

[6]  Henri Casanova,et al.  Measuring the Performance and Reliability of Production Computational Grids , 2006, 2006 7th IEEE/ACM International Conference on Grid Computing.

[7]  Alexandru Iosup,et al.  GRENCHMARK: A Framework for Analyzing, Testing, and Comparing Grids , 2006, Sixth IEEE International Symposium on Cluster Computing and the Grid (CCGRID'06).

[8]  Zsolt Németh,et al.  Performance evaluation on grids: directions, issues, and open problems , 2004, 12th Euromicro Conference on Parallel, Distributed and Network-Based Processing, 2004. Proceedings..

[9]  Alexandru Iosup,et al.  How are Real Grids Used? The Analysis of Four Grid Traces and Its Implications , 2006, 2006 7th IEEE/ACM International Conference on Grid Computing.