Discovering Piecewise Linear Models of Grid Workload

Despite extensive research focused on enabling QoS for grid users through economic and intelligent resource provisioning, no consensus has emerged on the most promising strategies. On top of intrinsically challenging problems, the complexity and size of data has so far drastically limited the number of comparative experiments. An alternative to experimenting on real, large, and complex data, is to look for well-founded and parsimonious representations. This study is based on exhaustive information about the gLite-monitored jobs from the EGEE grid, representative of a significant fraction of e-science computing activity in Europe. Our main contributions are twofold. First we found that workload models for this grid can consistently be discovered from the real data, and that limiting the range of models to piecewise linear time series models is sufficiently powerful. Second, we present a bootstrapping strategy for building more robust models from the limited samples at hand.

[1]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[2]  David Vengerov,et al.  A Reinforcement Learning Approach to Dynamic Resource Allocation ∗ , 2005 .

[3]  Lingyun Yang,et al.  Conservative Scheduling: Using Predicted Variance to Improve Scheduling Decisions in Dynamic Environments , 2003, ACM/IEEE SC 2003 Conference (SC'03).

[4]  B. Efron.  Bootstrap Methods: Another Look at the Jackknife , 1979 .

[5]  Yang Yang,et al.  Bagging binary and quantile predictors for time series , 2006 .

[6]  Fabrizio Gagliardi,et al.  Building an infrastructure for scientific Grid computing: status and goals of the EGEE project , 2005, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[7]  Warren Smith,et al.  Using Run-Time Predictions to Estimate Queue Wait Times and Improve Scheduler Performance , 1999, JSSPP.

[8]  J. Gott.  Implications of the Copernican principle for our future prospects , 1993, Nature.

[9]  Allen B. Downey,et al.  Using Queue Time Predictions for Processor Allocation , 1997, JSSPP.

[10]  Richard Wolski,et al.  Predicting Bounds on Queuing Delay in Space-shared Computing Environments , 2006, 2006 IEEE International Symposium on Workload Characterization.

[11]  Balázs Kégl,et al.  Utility-Based Reinforcement Learning for Reactive Grids , 2008, 2008 International Conference on Autonomic Computing.

[12]  Lorenza Saitta,et al.  Characterization of a computational grid as a complex system , 2009, GMAC '09.

[13]  Patrick Burns,et al.  Robustness of the Ljung-Box Test and its Rank Equivalent , 2002 .

[14]  Evgenia Smirni,et al.  Injecting realistic burstiness to a traditional client-server benchmark , 2009, ICAC '09.

[15]  Michèle Sebag,et al.  The Grid Observatory , 2011, 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[16]  Anand Raghunathan,et al.  Best-effort parallel execution framework for Recognition and mining applications , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[17]  Pascal Bondon,et al.  Discovering Linear Models of Grid Workload , 2009 .

[18]  Michael Muskulus,et al.  Analysis and modeling of job arrivals in a production grid , 2007, PERV.

[19]  Richard A. Davis,et al.  Structural Break Estimation for Nonstationary Time Series Models , 2006 .

[20]  Johan Montagnat,et al.  Modeling the latency on production grids with respect to the execution context , 2009, Parallel Comput..

[21]  Jordi Torres,et al.  Maximizing revenue in Grid markets using an economically enhanced resource manager , 2010, Concurr. Comput. Pract. Exp..

[22]  Rajarshi Das,et al.  On the use of hybrid reinforcement learning for autonomic resource allocation , 2007, Cluster Computing.

[23]  Jordi Torres,et al.  Maximizing revenue in Grid markets using an economically enhanced resource manager , 2010 .

[24]  Byoung-Dai Lee,et al.  Run-time prediction of parallel applications on shared environments , 2003, 2003 Proceedings IEEE International Conference on Cluster Computing.

[25]  Richard Wolski,et al.  Eliciting honest value information in a batch-queue environment , 2007, 2007 8th IEEE/ACM International Conference on Grid Computing.

[26]  Peter A. Dinda,et al.  Host load prediction using linear models , 2000, Cluster Computing.

[27]  Shantenu Jha,et al.  Investigating autonomic behaviours in grid-basedcomputational science applications , 2009, GMAC '09.

[28]  Alexandru Iosup,et al.  The Grid Workloads Archive , 2008, Future Gener. Comput. Syst..

[29]  Richard Wolski,et al.  Predicting the CPU availability of time‐shared Unix systems on the computational grid , 2004, Cluster Computing.

[30]  Michèle Sebag,et al.  Toward autonomic grids: analyzing the job flow with affinity streaming , 2009, KDD.

[31]  Remzi H. Arpaci-Dusseau,et al.  Gathering at the Well: Creating Communities for Grid I/O , 2001, ACM/IEEE SC 2001 Conference (SC'01).

[32]  Alexandru Iosup,et al.  Trace-based evaluation of job runtime and queue wait time predictions in grids , 2009, HPDC '09.

[33]  A. D. Meglio,et al.  Programming the Grid with gLite , 2006 .

[34]  Radu Prodan,et al.  Soft Benchmarks-Based Application Performance Prediction Using a Minimum Training Set , 2006, 2006 Second IEEE International Conference on e-Science and Grid Computing (e-Science'06).

[35]  Jorma Rissanen,et al.  Stochastic Complexity in Statistical Inquiry , 1989, World Scientific Series in Computer Science.