Performance characterization of heterogeneous distributed commodity cluster resources

While distributed computing systems which generally involve the aggregation of geographically distributed heterogeneous resources can in principle be used as computing platform, in practice, the discovery and selection of quality resources that satisfy user's application requirements remain an issue that is difficult to address. In this paper, we represent the performance of a computational cluster as a regression model that can be used to fine-tune the selection of suitable cluster resources. A study on the performance of distributed systems with respect to particular variations in parameters is presented. Our objective is to use a measurement-based evaluation technique to characterize the specific performance contribution of the individual cluster resource configurations. In the process we identify the key primary parameters (or factors) that should be considered when selecting and allocating a computational node for user application execution.

[1]  Douglas C. Montgomery,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[2]  Randy H. Katz,et al.  Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center , 2011, NSDI.

[3]  Richard Wolski,et al.  Dynamically forecasting network performance using the Network Weather Service , 1998, Cluster Computing.

[4]  Layuan Li,et al.  Pricing and Resource Allocation in Computational Grid with Utility Functions , 2005, ITCC.

[5]  Russell R. Barton,et al.  Ch. 7. A review of design and modeling in computer experiments , 2003 .

[6]  J. M. Lucas,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments, 3rd edition , 2010 .

[7]  Runze Li,et al.  Design of experiments with multiple independent variables: a resource management perspective on complete and reduced factorial designs. , 2009, Psychological methods.

[8]  Karen A. F. Copeland.  Experiments: Planning, Analysis, and Parameter Design Optimization , 2002 .

[9]  Robert W. Mee.  A Comprehensive Guide to Factorial Two-Level Experimentation , 2009 .

[10]  Chao-Tung Yang,et al.  Dynamic partitioning of loop iterations on heterogeneous PC clusters , 2007, The Journal of Supercomputing.

[11]  Chao-Tung Yang,et al.  A performance-based parallel loop scheduling on grid environments , 2007, The Journal of Supercomputing.

[12]  Richard Wolski,et al.  Predicting the CPU availability of time‐shared Unix systems on the computational grid , 2004, Cluster Computing.

[13]  Chao-Tung Yang,et al.  Using a Performance-based Skeleton to Implement Divisible Load Applications on Grid Computing Environments , 2009, J. Inf. Sci. Eng..

[14]  Los Angeles,et al.  Contributions in Design of Experiments: Methods and Applications , 2013 .

[15]  Peter A. Dinda,et al.  The statistical properties of host load , 1999, Sci. Program..

[16]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[17]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[18]  Yanmin Zhu,et al.  A Survey on Grid Scheduling Systems , 2013 .

[19]  Andrea C. Arpaci-Dusseau,et al.  The interaction of parallel and sequential workloads on a network of workstations , 1995, SIGMETRICS '95/PERFORMANCE '95.

[20]  Stephen A. Jarvis,et al.  Grid load balancing using intelligent agents , 2005, Future Gener. Comput. Syst..