An Adaptive Job Allocation Strategy for Heterogeneous Multiple Clusters

In this paper we proposal a new scheduling system based on multi-clusters environments. The work we present is Adaptive Job Allocation Strategy (AJAS) in which the scheduler dispatches jobs that with self-scheduling scheme into appropriate resources across multi-clusters. The strategy focuses on how to increase resource utility, jobs via self-scheduling and dispatching jobs to nodes with similar performance capacities, thus equalizing execution times among all the nodes the jobs require. Experimental results show that the proposed approach performs better than previous schemes.

[1]  Message Passing Interface Forum.  MPI: A message - passing interface standard , 1994 .

[2]  Miron Livny,et al.  Scheduling Mixed Workloads in Multi-grids: The Grid Execution Hierarchy , 2006, 2006 15th IEEE International Conference on High Performance Distributed Computing.

[3]  Thomas Sterling,et al.  How to Build a Beowulf: A Guide to the Implementation and Application of PC Clusters 2nd Printing , 1999 .

[4]  CONSTANTINE D. POLYCHRONOPOULOS,et al.  Guided Self-Scheduling: A Practical Scheduling Scheme for Parallel Supercomputers , 1987, IEEE Transactions on Computers.

[5]  Anthony T. Chronopoulos,et al.  A class of loop self-scheduling for heterogeneous clusters , 2001, Proceedings 42nd IEEE Symposium on Foundations of Computer Science.

[6]  Bu-Sung Lee,et al.  Workload management of cooperatively federated computing clusters , 2006, The Journal of Supercomputing.

[7]  Anca I. D. Bucur,et al.  Scheduling Policies for Processor Coallocation in Multicluster Systems , 2007, IEEE Transactions on Parallel and Distributed Systems.

[8]  Chao-Tung Yang,et al.  An Enhanced Parallel Loop Self-Scheduling Scheme for Cluster Environments , 2005, 19th International Conference on Advanced Information Networking and Applications (AINA'05) Volume 1 (AINA papers).

[9]  Rajkumar Buyya,et al.  High Performance Cluster Computing: Programming and Applications , 1999 .

[10]  Yi-Min Wang,et al.  Memory latency consideration for load sharing on heterogeneous network of workstations , 2006, J. Syst. Archit..

[11]  Daniel C. Stanzione,et al.  Characterization of Bandwidth-Aware Meta-Schedulers for Co-Allocating Jobs Across Multiple Clusters , 2005, The Journal of Supercomputing.

[12]  Minyi Guo,et al.  A taxonomy of application scheduling tools for high performance cluster computing , 2006, Cluster Computing.

[13]  Hui Li,et al.  Locality and Loop Scheduling on NUMA Multiprocessors , 1993, 1993 International Conference on Parallel Processing - ICPP'93.

[14]  Al Geist.  Cluster Computing: The Wave of the Future? , 1994, PARA.

[15]  Derek Wright,et al.  Cheap cycles from the desktop to the dedicated cluster: combining opportunistic and dedicated scheduling with Condor , 2007 .

[16]  Edith Schonberg,et al.  Factoring: a method for scheduling parallel loops , 1992 .

[17]  Rajkumar Buyya,et al.  High Performance Cluster Computing , 1999 .

[18]  Zhiyi Huang,et al.  Load Balancing in a Cluster Computer , 2006, 2006 Seventh International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT'06).

[19]  Jemal H. Abawajy,et al.  An efficient adaptive scheduling policy for high-performance computing , 2009, Future Gener. Comput. Syst..

[20]  Chao-Tung Yang,et al.  A parallel loop self-scheduling on grid computing environments , 2004, 7th International Symposium on Parallel Architectures, Algorithms and Networks, 2004. Proceedings..

[21]  Phillip Krueger,et al.  A comparison of preemptive and non-preemptive load distributing , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[22]  Michael J. Schulte,et al.  Memory latency consideration for load sharing on heterogeneous network of workstations , 2006 .

[23]  David E. Culler,et al.  A case for NOW (networks of workstation) , 1995, PODC '95.

[24]  Fernando Gustavo Tinetti,et al.  Parallel programming: techniques and applications using networked workstations and parallel computers. Barry Wilkinson, C. Michael Allen , 2000 .

[25]  L.M. Ni,et al.  Trapezoid Self-Scheduling: A Practical Scheduling Scheme for Parallel Compilers , 1993, IEEE Trans. Parallel Distributed Syst..

[26]  Ian T. Foster,et al.  The anatomy of the grid: enabling scalable virtual organizations , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[27]  Chao-Tung Yang,et al.  Well-Balanced Allocation Strategy for Multi-Cluster Computing Environments , 2008, 2008 12th IEEE International Workshop on Future Trends of Distributed Computing Systems.