Distributed Iterative Algorithms

Abstract We consider iterative algorithms of the form x := f(x) , executed by a distributed network of processors. We first discuss a few possibilities for synchronous execution. We then consider asynchronous implementations whereby each processor iterates on a different component of x , at its own pace, using the most recently received (but possibly outdated) information on the remaining components of x . While certain algorithms may fail to converge when implemented asynchronously, a large number of positive convergence results is available. We classify asynchronous algorithms into three main categories, depending on the amount of asynchronism they can tolerate, survey the corresponding convergence results, and discuss some applications

[1]  Dimitri P. Bertsekas,et al.  Distributed Asynchronous Relaxation Methods for Linear Network Flow Problems , 1987 .

[2]  F. Robert Contraction en norme vectorielle: Convergence d'iterations chaotiques pour des equations non linéaires de point fixe à plusieurs variables , 1976 .

[3]  H. Kushner,et al.  Asymptotic properties of distributed and communication stochastic approximation algorithms , 1987 .

[4]  Debasis Mitra,et al.  Asynchronous relaxations for the numerical solution of differential equations by parallel processors , 1985, PPSC.

[5]  D. Bertsekas,et al.  Distributed asynchronous optimal routing in data networks , 1984, The 23rd IEEE Conference on Decision and Control.

[6]  John N. Tsitsiklis,et al.  Problems in decentralized decision making and computation , 1984 .

[7]  H. Kushner,et al.  Stochastic approximation algorithms for parallel and distributed processing , 1987 .

[8]  Baruch Awerbuch An efficient network synchronization protocol , 1984, STOC '84.

[9]  Baruch Awerbuch,et al.  Complexity of network synchronization , 1985, JACM.

[10]  Gérard M. Baudet,et al.  Asynchronous Iterative Methods for Multiprocessors , 1978, JACM.

[11]  Debasis Mitra,et al.  A chaotic asynchronous algorithm for computing the fixed point of a nonnegative matrix of unit spectral radius , 1986, JACM.

[12]  Dimitri Bertsekas,et al.  Distributed dynamic programming , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[13]  Dimitri P. Bertsekas,et al.  Distributed asynchronous computation of fixed points , 1983, Math. Program..

[14]  John N. Tsitsiklis,et al.  Distributed asynchronous deterministic and stochastic gradient optimization algorithms , 1986 .

[15]  D. Bertsekas,et al.  Distributed asynchronous relaxation methods for convex network flow problems , 1987 .