Performance limits of divisible load processing in systems with limited communication buffers

In this work, we study influence of limited size of communication buffer on the efficiency of divisible loads processing. Divisible loads are computations which can be divided into parts of arbitrary sizes, and these parts can be processed in parallel. To finish processing in the shortest possible time an optimum distribution of the load must be calculated. The method of determining load distribution must take into account not only computing speed, but also interconnection system topology, communication medium speed and startup time. In this work, we include one more parameter: communication buffer size. We propose a general method of studying the influence of the communication buffer size on the interaction between the communication and computations. Three archetypal interconnection topologies are examined: stars, ordinary trees, and binomial trees. The results of modeling the performance of parallel systems show that the influence of communication buffer size is indirect and qualitative in nature. Buffer size affects the performance by causing message fragmentation, or changing load balance among the processors. We analyze performance of several communication algorithms and their interaction with the computations. The simulations show that these classic algorithms are limited.

[1]  F. Leighton,et al.  Introduction to Parallel Algorithms and Architectures: Arrays, Trees, Hypercubes , 1991 .

[2]  Maciej Drozdowski,et al.  Selected problems of scheduling tasks in multiprocessor computer systems , 1997 .

[3]  Frank Thomson Leighton.  Introduction to parallel algorithms and architectures: arrays , 1992 .

[4]  Debasish Ghose,et al.  Divisible Load Theory: A New Paradigm for Load Scheduling in Distributed Systems , 2004, Cluster Computing.

[5]  Thomas G. Robertazzi,et al.  Distributed computation with communication delay (distributed intelligent sensor networks) , 1988 .

[6]  Jacek Blazewicz,et al.  Scheduling a Divisible Task in a Two-dimensional Toroidal Mesh , 1999, Discret. Appl. Math..

[7]  Pawel Wolniewicz,et al.  Experiments with Scheduling Divisible Tasks in Clusters of Workstations , 2000, Euro-Par.

[8]  Debasish Ghose,et al.  Scheduling Divisible Loads in Parallel and Distributed Systems , 1996 .

[9]  Laurence A. Wolsey,et al.  Integer and Combinatorial Optimization , 1988 .

[10]  Joseph G. Peters,et al.  Circuit-Switched Broadcasting in Torus Networks , 1996, IEEE Trans. Parallel Distributed Syst..

[11]  Thomas G. Robertazzi,et al.  Ten Reasons to Use Divisible Load Theory , 2003, Computer.

[12]  Jacek Blazewicz,et al.  Divisible task scheduling - Concept and verification , 1999, Parallel Comput..

[13]  Gerassimos D. Barlas.  Collection-Aware Optimum Sequencing of Operations and Closed-Form Solutions for the Distribution of a Divisible Load on Arbitrary Processor Trees , 1998, IEEE Trans. Parallel Distributed Syst..

[14]  Bharadwaj Veeravalli,et al.  Access Time Minimization for Distributed Multimedia Applications , 2000, Multimedia Tools and Applications.