Grid job scheduling using Route with Genetic Algorithm support

In 2006 the Route load balancing algorithm was proposed and compared to other techniques aiming at optimizing the process allocation in grid environments. This algorithm schedules tasks of parallel applications considering computer neighborhoods (where the distance is defined by the network latency). Route presents good results for large environments, although there are cases where neighbors do not have an enough computational capacity nor communication system capable of serving the application. In those situations the Route migrates tasks until they stabilize in a grid area with enough resources. This migration may take long time what reduces the overall performance. In order to improve such stabilization time, this paper proposes RouteGA (Route with Genetic Algorithm support) which considers historical information on parallel application behavior and also the computer capacities and load to optimize the scheduling. This information is extracted by using monitors and summarized in a knowledge base used to quantify the occupation of tasks. Afterwards, such information is used to parameterize a genetic algorithm responsible for optimizing the task allocation. Results confirm that RouteGA outperforms the load balancing carried out by the original Route, which had previously outperformed others scheduling algorithms from literature.

[1]  Arnold Robbins,et al.  Linux in a Nutshell (In a Nutshell (O'Reilly)) , 2005 .

[2]  Luciano José Senger,et al.  A new migration model based on the evaluation of processes load and lifetime on heterogeneous computing environments , 2004 .

[3]  Richard Gibbons,et al.  A Historical Application Profiler for Use by Parallel Schedulers , 1997, JSSPP.

[4]  Roger W. Hockney.  The science of computer benchmarking , 1995, Software, environments, tools.

[5]  Laurence T. Yang,et al.  RouteGA: A Grid Load Balancing Algorithm with Genetic Support , 2007, 21st International Conference on Advanced Information Networking and Applications (AINA '07).

[6]  Mark Baker,et al.  Cluster Computing White Paper , 2000, ArXiv.

[7]  Francine Berman,et al.  A study of deadline scheduling for client-server systems on the Computational Grid , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.

[8]  Arjan J. C. van Gemund,et al.  LLB: A fast and effective scheduling algorithm for distributed-memory systems , 1999, Proceedings 13th International Parallel Processing Symposium and 10th Symposium on Parallel and Distributed Processing. IPPS/SPDP 1999.

[9]  Evgueni Dodonov,et al.  GridBox: securing hosts from malicious and greedy applications , 2004, MGC '04.

[10]  R. Hinterding,et al.  Representation, mutation and crossover issues in evolutionary computation , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[11]  Tim Brecht,et al.  Using Parallel Program Characteristics in Dynamic Processor Allocation Policies , 1996, Perform. Evaluation.

[12]  Rodrigo Fernandes de Mello,et al.  Model for Simulation of Heterogeneous High-Performance Computing Environments , 2006, VECPAR.

[13]  Tony R. Martinez,et al.  Improved Heterogeneous Distance Functions , 1996, J. Artif. Intell. Res..

[14]  Rajkumar Buyya,et al.  GridSim: a toolkit for the modeling and simulation of distributed resource management and scheduling for Grid computing , 2002, Concurr. Comput. Pract. Exp..

[15]  Kenneth C. Sevcik.  Characterizations of parallelism in applications and their use in scheduling , 1989, SIGMETRICS '89.

[16]  Henri Casanova,et al.  A realistic network/application model for scheduling divisible loads on large-scale platforms , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[17]  Henri Casanova,et al.  Parameter Sweeps on the Grid with APST , 2003 .

[18]  Arnold Robbins,et al.  Linux in a Nutshell , 1997 .

[19]  Dharma P. Agrawal,et al.  A task duplication based scheduling algorithm for heterogeneous systems , 2000, Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000.

[20]  Laurence T. Yang,et al.  Comparative study of the server-initiated lowest algorithm using a load balancing index based on the process behavior for heterogeneous environment , 2006, Cluster Computing.

[21]  Mark S. Squillante,et al.  Processor Allocation in Multiprogrammed Distributed-Memory Parallel Computer Systems , 1997, J. Parallel Distributed Comput..

[22]  Francine Berman,et al.  Heuristics for scheduling parameter sweep applications in grid environments , 2000, Proceedings 9th Heterogeneous Computing Workshop (HCW 2000) (Cat. No.PR00556).

[23]  Allen B. Downey.  Predicting queue times on space-sharing parallel computers , 1997, Proceedings 11th International Parallel Processing Symposium.

[24]  Henri Casanova,et al.  Practical divisible load scheduling on grid platforms with APST-DV , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[25]  Dror G. Feitelson,et al.  Improved Utilization and Responsiveness with Gang Scheduling , 1997, JSSPP.

[26]  Laurence T. Yang,et al.  A routing load balancing policy for grid computing environments , 2006, 20th International Conference on Advanced Information Networking and Applications - Volume 1 (AINA'06).

[27]  Marcos José Santana,et al.  Scheduling based on the impact over process communication of parallel applications , 2005, 19th International Symposium on High Performance Computing Systems and Applications (HPCS'05).

[28]  D. Fogel,et al.  Basic Algorithms and Operators , 1999 .

[29]  Francine Berman,et al.  Grid Computing: Making the Global Infrastructure a Reality , 2003 .

[30]  Fabio Kon,et al.  Proceedings of the 2nd workshop on Middleware for grid computing , 2004 .

[31]  Domenico Ferrari,et al.  An Experimental Study of Load Balancing Performance , 1987 .

[32]  Mukesh Singhal,et al.  Load distributing for locally distributed systems , 1992, Computer.

[33]  D. Fogel,et al.  Advanced Algorithms and Operators , 1999 .

[34]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[35]  Warren Smith,et al.  Using Run-Time Predictions to Estimate Queue Wait Times and Improve Scheduler Performance , 1999, JSSPP.

[36]  Lee C. Potter,et al.  Statistical Prediction of Task Execution Times through Analytic Benchmarking for Scheduling in a Heterogeneous Environment , 1999, IEEE Trans. Computers.

[37]  Rajkumar Buyya,et al.  High Performance Cluster Computing: Architectures and Systems , 1999 .

[38]  Tae-Young Choe,et al.  A task duplication based scheduling algorithm with optimality condition in heterogeneous systems , 2002, Proceedings. International Conference on Parallel Processing Workshop.

[39]  Yang Gao,et al.  Adaptive grid job scheduling with genetic algorithms , 2005, Future Gener. Comput. Syst..