Classification of Task Partitioning and Load Balancing Strategies in Distributed Parallel Computing Systems

Distribution of the tasks amongst the various computing nodes is itself an intellectually challenging problem in the high performance distributed computing systems. To choose the appropriate strategy for the required system is difficult without the meaningful comparison of the existing task partitioning and load balancing strategies. The effectiveness of the strategy depend on the number of factors-efficiency, interconnection topology, communication mode, program structure, throughput and computing capabilities of the structure. A number of task partitioning and load balancing strategies have been proposed, each of which perform remarkable results under different circumstances. The main goal of the paper is to unravel the mystery of strategies and to classify when and where each strategy is appropriate. In this paper, taxonomy of task partitioning and load balancing is presented in an attempt to provide a common terminology and classification mechanism.

[1]  Ishfaq Ahmad,et al.  Efficient Scheduling of Arbitrary TAsk Graphs to Multiprocessors Using a Parallel Genetic Algorithm , 1997, J. Parallel Distributed Comput..

[2]  Arjan J. C. van Gemund,et al.  On the complexity of list scheduling algorithms for distributed-memory systems , 1999, ICS '99.

[3]  Jan Karel Lenstra,et al.  A classification scheme for vehicle routing and scheduling problems , 1990 .

[4]  Masud Hasan,et al.  Fast Preemptive Task Scheduling Algorithm for Homogeneous and Heterogeneous Distributed Memory Systems , 2008, 2008 Ninth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing.

[5]  Daniela Zaharie,et al.  Evolutionary task scheduling in static and dynamic environments , 2010, 2010 International Joint Conference on Computational Cybernetics and Technical Informatics.

[6]  Weimin Zheng,et al.  A DAG-based partitioning-reconfiguring scheduling algorithm in network of workstations , 2000, Proceedings Fourth International Conference/Exhibition on High Performance Computing in the Asia-Pacific Region.

[7]  Howard Jay Siegel,et al.  Task execution time modeling for heterogeneous computing systems , 2000, Proceedings 9th Heterogeneous Computing Workshop (HCW 2000) (Cat. No.PR00556).

[8]  Daniel Gajski,et al.  Hypertool: A Programming Aid for Message-Passing Systems , 1990, IEEE Trans. Parallel Distributed Syst..

[9]  Mihalis Yannakakis,et al.  Towards an architecture-independent analysis of parallel algorithms , 1990, STOC '88.

[10]  Hsin-An Chen.  On the design of task scheduling in the heterogeneous computing environments , 2005, PACRIM. 2005 IEEE Pacific Rim Conference on Communications, Computers and signal Processing, 2005..

[11]  Vivek Sarkar,et al.  Partitioning and Scheduling Parallel Programs for Multiprocessing , 1989 .

[12]  M. Tahar Kechadi,et al.  Dynamic task scheduling in computing cluster environments , 2004, Third International Symposium on Parallel and Distributed Computing/Third International Workshop on Algorithms, Models and Tools for Parallel Computing on Heterogeneous Networks.

[13]  Alan Burns,et al.  Preemptive priority-based scheduling: an appropriate engineering approach , 1995 .

[14]  Imtiaz Ahmad,et al.  DPS: dynamic priority scheduling heuristic for heterogeneous computing systems , 1998 .

[15]  D Parkinson,et al.  High–Speed Computing , 1978 .

[16]  Michael G. Norman,et al.  Models of machines and computation for mapping in multicomputers , 1993, CSUR.

[17]  Edward A. Lee,et al.  A Compile-Time Scheduling Heuristic for Interconnection-Constrained Heterogeneous Processor Architectures , 1993, IEEE Trans. Parallel Distributed Syst..

[18]  Frank D. Anger,et al.  Scheduling Precedence Graphs in Systems with Interprocessor Communication Times , 1989, SIAM J. Comput..

[19]  Dimitrios G. Pandelis,et al.  Optimal preemptive scheduling on uniform machines with discounted flowtime objectives , 2007, Eur. J. Oper. Res..

[20]  Satish K. Tripathi,et al.  Static and Dynamic Processor Scheduling Disciplines in Heterogeneous Parallel Architectures , 1995, J. Parallel Distributed Comput..

[21]  Charles U. Martel,et al.  On non-preemptive scheduling of period and sporadic tasks , 1991, [1991] Proceedings Twelfth Real-Time Systems Symposium.

[22]  Füsun Özgüner,et al.  Parallelizing Existing Applications in a Distributed Heterogeneous Environment , 1995 .

[23]  Y.-K. Kwok,et al.  Static scheduling algorithms for allocating directed task graphs to multiprocessors , 1999, CSUR.

[24]  JOHN B. ANDREWS,et al.  An Analytical Approach to Performance/Cost Modeling of Parallel Computers , 1991, J. Parallel Distributed Comput..

[25]  Gary E. Christensen.  MIMD vs. SIMD Parallel Processing: A Case Study in 3D Medical Image Registration , 1998, Parallel Comput..

[26]  Marco Furini,et al.  International Journal of Computer and Applications , 2010 .

[27]  David P. Williamson,et al.  Scheduling Parallel Machines On-Line , 1995, SIAM J. Comput..

[28]  Björn Andersson,et al.  Preemptive multiprocessor scheduling anomalies , 2002, Proceedings 16th International Parallel and Distributed Processing Symposium.

[29]  Larry Rudolph,et al.  Parallel Job Scheduling: Issues and Approaches , 1995, JSSPP.

[30]  Michael F. P. O'Boyle,et al.  A Static Task Partitioning Approach for Heterogeneous Systems Using OpenCL , 2011, CC.

[31]  R. A. Starke,et al.  A Heterogeneous Preemptive and Non-preemptive Scheduling Approach for Real-Time Systems on Multiprocessors , 2012, 2012 Second Brazilian Conference on Critical Embedded Systems.