A survey on methodologies for runtime prediction on grid environments

The accurate prediction of runtimes of future job tasks on the nodes of a grid supplies vital information for the users to make CPU resource usage decisions. There are number of different approaches to predict runtimes of the future job tasks. These approaches range from the statistical to non statistical and some of them require expensive search algorithms or availability of the source code of job tasks. In this paper we discuss about the existing such methods and categorise them according to a certain taxonomy. Then we compare the advantages and disadvantages of them with that of the Task Profiling Model for Host Load Profile Prediction which is developed by the authors.

[1]  Frédéric Suter,et al.  Dynamic Performance Forecasting for Network-Enabled Servers in a Heterogeneous Environment , 2001 .

[2]  Max D. Morris.  The Design and Analysis of Computer Experiments. Thomas J. Santner , Brian J. Williams , and William I. Notz , 2004 .

[3]  David C. Levy,et al.  Task profiling model for load profile prediction , 2011, Future Gener. Comput. Syst..

[4]  M. Morris.  The Design and Analysis of Computer Experiments , 2004 .

[5]  Ian Foster,et al.  Predicting application run times with historical information , 2004, J. Parallel Distributed Comput..

[6]  Jan Broeckhove,et al.  Runtime Prediction Based Grid Scheduling of Parameter Sweep Jobs , 2008, 2008 IEEE Asia-Pacific Services Computing Conference.

[7]  Andrzej Skowron,et al.  Rough-Fuzzy Hybridization: A New Trend in Decision Making , 1999 .

[8]  Hui Li,et al.  Mining performance data for metascheduling decision support in the Grid , 2007, Future Gener. Comput. Syst..

[9]  Gagan Agrawal,et al.  A Performance Prediction Framework for Grid-Based Data Mining Applications , 2007, 2007 IEEE International Parallel and Distributed Processing Symposium.

[10]  Qiang Xu,et al.  Performance prediction with skeletons , 2008, Cluster Computing.

[11]  Hong Linh Truong,et al.  ASKALON: a tool set for cluster and Grid computing: Research Articles , 2005 .

[12]  Dror G. Feitelson,et al.  The workload on parallel supercomputers: modeling the characteristics of rigid jobs , 2003, J. Parallel Distributed Comput..

[13]  H. Lilliefors.  On the Kolmogorov-Smirnov Test for the Exponential Distribution with Mean Unknown , 1969 .

[14]  Jesús Labarta,et al.  eNANOS: Coordinated Scheduling in Grid Environments , 2005, PARCO.

[15]  Shonali Krishnaswamy,et al.  Estimating computation times in data intensive e-services , 2003, Proceedings of the Fourth International Conference on Web Information Systems Engineering, 2003. WISE 2003..

[16]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[17]  Füsun Özgüner,et al.  Run-time statistical estimation of task execution times for heterogeneous distributed computing , 1996, Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing.

[18]  Jianqing Fan,et al.  Local polynomial modelling and its applications , 1994 .

[19]  Tony R. Martinez,et al.  Improved Heterogeneous Distance Functions , 1996, J. Artif. Intell. Res..

[20]  Jesús Labarta,et al.  Multi-Criteria Grid Resource Management Using Performance Prediction Techniques , 2007 .

[21]  Richard Wolski,et al.  Dynamically forecasting network performance using the Network Weather Service , 1998, Cluster Computing.

[22]  Matthias S. Müller,et al.  Performance Prediction in a Grid Environment , 2003, European Across Grids Conference.

[23]  Radu Prodan,et al.  ASKALON: a tool set for cluster and Grid computing , 2005, Concurr. Pract. Exp..

[24]  Thomas Rauber,et al.  A source code analyzer for performance prediction , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[25]  Chris Peterson,et al.  Implementing a Performance Forecasting System for Metacomputing The Network Weather Service , 1997, ACM/IEEE SC 1997 Conference (SC'97).

[26]  Byoung-Dai Lee,et al.  Run-time prediction of parallel applications on shared environments , 2003, 2003 Proceedings IEEE International Conference on Cluster Computing.

[27]  N Ranaldo,et al.  Parallel Computing: Current & Future Issues of High-End Computing , 2006 .

[28]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[29]  Richard McClatchey,et al.  Predicting the Resource Requirements of a Job Submission , 2004 .

[30]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[31]  Kai Hwang,et al.  Adaptive Workload Prediction of Grid Performance in Confidence Windows , 2010, IEEE Transactions on Parallel and Distributed Systems.

[32]  Luís Moura Silva,et al.  Predicting Machine Availabilities in Desktop Pools , 2006, 2006 IEEE/IFIP Network Operations and Management Symposium NOMS 2006.

[33]  Andrew D. Back,et al.  Radial Basis Functions , 2001 .

[34]  Graham R. Nudd,et al.  Pace—A Toolset for the Performance Prediction of Parallel and Distributed Systems , 2000, Int. J. High Perform. Comput. Appl..

[35]  Peter A. Dinda,et al.  Resource Signal Prediction and Its Application to Real-time Scheduling Advisors (Thesis Summary) , 2000 .

[36]  David Abramson,et al.  Neuroscience instrumentation and distributed analysis of brain activity data: a case for eScience on global Grids , 2005, Concurr. Comput. Pract. Exp..

[37]  Tong Zhang,et al.  An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods , 2001, AI Mag..

[38]  Warren Smith,et al.  Using Run-Time Predictions to Estimate Queue Wait Times and Improve Scheduler Performance , 1999, JSSPP.

[39]  Mor Harchol-Balter,et al.  Exploiting process lifetime distributions for dynamic load balancing , 1995, SIGMETRICS.

[40]  Andrew W. Moore,et al.  Locally Weighted Learning , 1997, Artificial Intelligence Review.

[41]  Ramin Yahyapour,et al.  Parallel Computer Workload Modeling with Markov Chains , 2004, JSSPP.

[42]  Lingyun Yang,et al.  Conservative Scheduling: Using Predicted Variance to Improve Scheduling Decisions in Dynamic Environments , 2003, ACM/IEEE SC 2003 Conference (SC'03).