Effects of Asynchronism on the Convergence Rate of Iterative Algorithms

In multiprocessor systems, iterative algorithms can be implemented synchronously or asynchronously. Unfortunately, few guidelines exist to make a choice. In this paper, we compare the execution times of an asynchronous iterative algorithm and of its synchronous counterpart. Synchronization overhead and communication times are neglected in order to focus on the effect of asynchronism on the convergence rate. Under some assumptions, we derive an analytical model. In this model,Qtasks with identical and independently distributed execution time distributions execute onPprocessors. We show the effects of execution time fluctuations, of the number of processors and tasks, of the scheduling policy and of the amount of coupling among iterate components. The models show that the asynchronous iteration may be up to twice as slow as the synchronous one. This worst case is achieved when the processing time fluctuations are very small, and the coupling among components is strong.

[1]  A. Frommer.  On asynchronous iterations in partially ordered spaces , 1991 .

[2]  Gérard M. Baudet,et al.  Asynchronous Iterative Methods for Multiprocessors , 1978, JACM.

[3]  Timothy S. Axelrod,et al.  Effects of synchronization barriers on multiprocessor performance , 1986, Parallel Comput..

[4]  D. E. Baz.  M -functions and parallel asynchronous algorithms , 1990 .

[5]  J. Mark Bull,et al.  Numerical Performance of an Asynchronous Jacobi Iteration , 1992, CONPAR.

[6]  R M Haralick,et al.  The consistent labeling problem: part I. , 1979, IEEE transactions on pattern analysis and machine intelligence.

[7]  J. Miellou,et al.  Un critère de convergence pour des méthodes générales de point fixe , 1985 .

[8]  M. Tarazi.  Some convergence results for asynchronous algorithms , 1982 .

[9]  Michel Dubois,et al.  Parallel asynchronous algorithms for discrete data , 1990, JACM.

[10]  Michel Dubois,et al.  Sufficient conditions for the convergence of asynchronous iterations , 1989, Parallel Computing.

[11]  Jean-Pierre Prost,et al.  Synchronization and Load Unbalance Effects of Parallel Iterative Algorithms , 1989, ICPP.

[12]  Alan Weiss,et al.  Allocating Independent Subtasks on Parallel Processors , 1985, IEEE Transactions on Software Engineering.

[13]  Dimitri Bertsekas,et al.  Distributed dynamic programming , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[14]  Anne Greenbaum.  Synchronization costs on multiprocessors , 1989, Parallel Comput..

[15]  Dimitri P. Bertsekas,et al.  Distributed asynchronous computation of fixed points , 1983, Math. Program..

[16]  John N. Tsitsiklis,et al.  On the average communication complexity of asynchronous distributed algorithms , 1995, JACM.

[17]  Michel Dubois,et al.  The Run-Time Efficiency of Parallel Asynchronous Algorithms , 1991, IEEE Trans. Computers.

[18]  J. T. Robinson,et al.  Some Analysis Techniques for Asynchronous Multiprocessor Algorithms , 1979, IEEE Transactions on Software Engineering.

[19]  John N. Tsitsiklis,et al.  Convergence rate and termination of asynchronous iterative algorithms , 1989, ICS '89.

[20]  Michel Dubois,et al.  Performance of Synchronized Iterative Processes in Multiprocessor Systems , 1982, IEEE Transactions on Software Engineering.

[21]  J. C. Miellou.  Asynchronous iterations and order intervals , 1986 .