Dynamic Task Allocation Models for Large Distributed Computing Systems

Dynamic task allocation for distributed computing systems (DCS) is an important goal to be achieved for engineering applications. The purpose of dynamic task allocation is to increase the system throughput in a dynamic environment, which can be done by balancing the utilization of computing resources and minimizing communication between processors during run time. In this paper, we propose two dynamic task allocation models which are: 1) the clustering simulated annealing model (CSAM); and 2) the mean field annealing model (MFAM). Both of these models combine characteristics of statistical and deterministic approaches. These models provide the rapid convergence characteristic of the deterministic approaches while preserving the solution quality afforded by simulated annealing. Simulation results of the CSAM and MFAM provide a stable and balanced system with 50% and 10% of the convergence time needed by simulated annealing, respectively. The results of this research are important in that it presents the feasibility of applying statistically based task allocation models on large DCSs in a dynamic environment. Solutions of these models depend on the annealing process instead of the structures of the input data, providing the possibility of obtaining better solutions by using more efficient computing hardware.

[1]  Scott F. Midkiff,et al.  Heuristic Technique for Processor and Link Assignment in Multicomputers , 1991, IEEE Trans. Computers.

[2]  Edward D. Lazowska,et al.  Speedup Versus Efficiency in Parallel Systems , 1989, IEEE Trans. Computers.

[3]  José M. F. Moura,et al.  Parallel processing on supercomputers: a set of computational experiments , 1988, Proceedings. SUPERCOMPUTING '88.

[4]  Miller,et al.  Graph partitioning using annealed neural networks , 1989 .

[5]  Harold S. Stone,et al.  Multiprocessor Scheduling with the Aid of Network Flow Algorithms , 1977, IEEE Transactions on Software Engineering.

[6]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[7]  Thomas L. Casavant,et al.  Effects of Response and Stability on Scheduling in Distributed Computing Systems , 1988, IEEE Trans. Software Eng..

[8]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Tevfik Bultan,et al.  A New Mapping Heuristic Based on Mean Field Annealing , 1992, J. Parallel Distributed Comput..

[10]  Jacob A. Abraham,et al.  Load Balancing in Distributed Systems , 1982, IEEE Transactions on Software Engineering.

[11]  Masahiro Tsuchiya,et al.  A Task Allocation Model for Distributed Computing Systems , 1982, IEEE Transactions on Computers.

[12]  J. M. Swisshelm Multitasking a three-dimensional Navier-Stokes algorithm on the Cray-2 , 1988, Proceedings Supercomputing Vol.II: Science and Applications.

[13]  G. C. Fox,et al.  Solving Problems on Concurrent Processors , 1988 .

[14]  Geoffrey C. Fox,et al.  Monte Carlo physics on a concurrent processor , 1986 .

[15]  Kemal Efe,et al.  Heuristic Models of Task Assignment Scheduling in Distributed Systems , 1982, Computer.

[16]  David M. Nicol,et al.  Optimal Dynamic Remapping of Data Parallel Computations , 1990, IEEE Trans. Computers.

[17]  A. J. Herbert The Cambridge Distributed Computing System , 1983, Advanced Course: Local Area Networks.

[18]  William Stallings,et al.  Data and Computer Communications , 1985 .

[19]  Janak H. Patel,et al.  Load balancing and task decomposition techniques for parallel implementation of integrated vision systems algorithms , 1989, Proceedings of the 1989 ACM/IEEE Conference on Supercomputing (Supercomputing '89).

[20]  Carsten Peterson,et al.  Explorations of the mean field theory learning algorithm , 1989, Neural Networks.

[21]  S. Arunkumar,et al.  A randomized heuristics for the mapping problem: The genetic approach , 1992, Parallel Comput..

[22]  Jonathan Schaeffer,et al.  A Template-Based Approach to the Generation of Distributed Applications Using a Network of Workstations , 1991, IEEE Trans. Parallel Distributed Syst..

[23]  Wesley W. Chu,et al.  Task Allocation in Distributed Data Processing , 1980, Computer.

[24]  Michael McLaughlin Simulated annealing , 1989 .

[25]  J. Pasquale Using expert systems to manage distributed computer systems , 1988, IEEE Network.

[26]  Johan De Keyser,et al.  Load Balancing Data Parallel Programs on Distributed Memory Computers , 1993, Parallel Comput..

[27]  Alice Koniges,et al.  Parallel computing and multitasking , 1986 .

[28]  Wesley W. Chu,et al.  Task Allocation and Precedence Relations for Distributed Real-Time Systems , 1987, IEEE Transactions on Computers.

[29]  Michael J. Fischer,et al.  The architecture of the Eden system , 1981, SOSP.

[30]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[31]  Thomas L. Casavant,et al.  A Taxonomy of Scheduling in General-Purpose Distributed Computing Systems , 1988, IEEE Trans. Software Eng..

[32]  Kang G. Shin,et al.  Load Sharing in Distributed Real-Time Systems with State-Change Broadcasts , 1989, IEEE Trans. Computers.

[33]  Geoffrey C. Fox,et al.  Concurrent computation and the theory of complex systems , 1986 .

[34]  Greg Thiel,et al.  LOCUS a network transparent, high reliability distributed system , 1981, SOSP.

[35]  Chien-Chung Shen,et al.  A Graph Matching Approach to Optimal Task Assignment in Distributed Computing Systems Using a Minimax Criterion , 1985, IEEE Trans. Computers.

[36]  R. Morison,et al.  The scattered decomposition for finite elements , 1987 .

[37]  Roberto Serra,et al.  Complex Systems and Cognitive Processes , 1990, Springer Berlin Heidelberg.

[38]  John A. Stankovic,et al.  An Application of Bayesian Decision Theory to Decentralized Control of Job Scheduling , 1985, IEEE Transactions on Computers.

[39]  John A. Stankovic,et al.  An Adaptive Bidding Algorithm For Processes, Clusters and Distributed Groups , 1984, ICDCS.

[40]  A. J. Harget,et al.  Load balancing algorithms in loosely-coupled distributed systems: a survey , 1990 .

[41]  David R. Cheriton,et al.  The V distributed system , 1988, CACM.

[42]  John A. Stankovic,et al.  A Perspective on Distributed Computer Systems , 1984, IEEE Transactions on Computers.

[43]  Reid G. Smith,et al.  The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver , 1980, IEEE Transactions on Computers.

[44]  Geoffrey C. Fox,et al.  A Review of Automatic Load Balancing and Decomposition Methods for the Hypercube , 1988 .

[45]  Virginia Mary Lo,et al.  Heuristic Algorithms for Task Assignment in Distributed Systems , 1988, IEEE Trans. Computers.