ServMark: A Framework for Testing Grid Services

The dynamicity, the heterogeneity, or simply the sheer scale of today’s grids expose in grid services problems of performance, functionality, and reliability. While testing under realistic conditions is a proven industrial method, little has been done in grids in this direction. Early testing in real grids reveals failure rates from 10% and up to 45% [1], [2], [3], and functionality problems of around 1 every 3 tests for widelyinstalled grid services [4]1. To further the development of grid services testing in large-scale settings, we present in this work the SERVMARK framework. In SERVMARK, we address two orthogonal research questions : (1) How to test a large-scale, distributed, and (grid)service-based environment? and (2) How to generate realistic testing traces for a wide-range of testing scenarios?. To the best of our knowledge, and as the main contribution of this work, ours is the first approach that answers both questions in the context of functionality testing and system tuning, of performance testing, and of reliability testing.

[1]  Ian T. Foster,et al.  DiPerF: an automated distributed performance testing framework , 2004, Fifth IEEE/ACM International Workshop on Grid Computing.

[2]  Hui Li,et al.  Job Failure Analysis and Its Implications in a Large-Scale Production Grid , 2006, 2006 Second IEEE International Conference on e-Science and Grid Computing (e-Science'06).

[3]  Alexandru Iosup,et al.  GRENCHMARK: A Framework for Analyzing, Testing, and Comparing Grids , 2006, Sixth IEEE International Symposium on Cluster Computing and the Grid (CCGRID'06).

[4]  Alexandru Iosup,et al.  Build-and-Test Workloads for Grid Middleware: Problem, Analysis, and Applications , 2007, Seventh IEEE International Symposium on Cluster Computing and the Grid (CCGrid '07).

[5]  Bianca Schroeder,et al.  A Large-Scale Study of Failures in High-Performance Computing Systems , 2006, IEEE Transactions on Dependable and Secure Computing.

[6]  Henri Casanova,et al.  Measuring the Performance and Reliability of Production Computational Grids , 2006, 2006 7th IEEE/ACM International Conference on Grid Computing.