Adaptive multi-resource prediction in distributed resource sharing environment

Resource prediction can greatly assist resource selection and scheduling in a distributed resource sharing environment such as a computational Grid. Existing resource prediction models are either based on the auto-correlation of a single resource or based on the cross correlation between two resources. In this paper, we propose a multi-resource prediction model (MModel) that uses both kinds of correlations to achieve higher prediction accuracy. We also present two adaptation techniques that enable the MModel to adapt to the time-varying characteristics of the underlying resources. Experimental results with CPU load prediction in both workstation and Grid environment show that on average, the adaptive MModel (called MModel-a) can achieve from 6% to more than 96% reduction in prediction errors compared with the autoregressive (AR) model, which has previously been shown to work well for CPU load predictions.

[1]  Ian T. Foster,et al.  Homeostatic and tendency-based CPU load predictions , 2003, Proceedings International Parallel and Distributed Processing Symposium.

[2]  Lingyun Yang,et al.  Conservative Scheduling: Using Predicted Variance to Improve Scheduling Decisions in Dynamic Environments , 2003, ACM/IEEE SC 2003 Conference (SC'03).

[3]  Francine Berman,et al.  Scheduling from the perspective of the application , 1996, Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing.

[4]  Peter A. Dinda,et al.  The statistical properties of host load , 1999, Sci. Program..

[5]  WolskiRich.  Dynamically forecasting network performance using the Network Weather Service , 1998 .

[6]  Ian T. Foster,et al.  The anatomy of the grid: enabling scalable virtual organizations , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[7]  Richard Wolski,et al.  The network weather service: a distributed resource performance forecasting service for metacomputing , 1999, Future Gener. Comput. Syst..

[8]  Richard Wolski,et al.  Multivariate Resource Performance Forecasting in the Network Weather Service , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[9]  Francine Berman,et al.  A Decoupled Scheduling Approach for the GrADS Program Development Environment , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[10]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[11]  Jennifer M. Schopf,et al.  Using Disk Throughput Data in Predictions of End-to-End Grid Data Transfers , 2002, GRID.

[12]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[13]  Peter A. Dinda,et al.  An evaluation of linear models for host load prediction , 1999, Proceedings. The Eighth International Symposium on High Performance Distributed Computing (Cat. No.99TH8469).

[14]  Richard Wolski,et al.  Predicting the CPU availability of time‐shared Unix systems on the computational grid , 2004, Cluster Computing.

[15]  Peter A. Dinda,et al.  Host load prediction using linear models , 2000, Cluster Computing.

[16]  Jennifer M. Schopf,et al.  Predicting sporadic grid data transfers , 2002, Proceedings 11th IEEE International Symposium on High Performance Distributed Computing.

[17]  R. Wolski,et al.  Predicting the CPU availability of time‐shared Unix systems on the computational grid , 1999, Proceedings. The Eighth International Symposium on High Performance Distributed Computing (Cat. No.99TH8469).

[18]  Ian Foster,et al.  The Grid: A New Infrastructure for 21st Century Science , 2002 .

[19]  Richard Wolski,et al.  Dynamically forecasting network performance using the Network Weather Service , 1998, Cluster Computing.