论文信息 - Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms

Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms

We present a model for asynchronous distributed computation and then proceed to analyze the convergence of natural asynchronous distributed versions of a large class of deterministic and stochastic gradient-like algorithms. We show that such algorithms retain the desirable convergence properties of their centralized counterparts, provided that the time between consecutive communications between processors plus communication delays are not too large.

[1] P. Meyer. Probability and potentials , 1966 .

[2] H. T. Kung,et al. Synchronized and asynchronous parallel algorithms for multiprocessors , 1976 .

[3] Robert G. Gallager,et al. A Minimum Delay Routing Algorithm Using Distributed Computation , 1977, IEEE Trans. Commun..

[4] Gérard M. Baudet,et al. Asynchronous Iterative Methods for Multiprocessors , 1978, JACM.

[5] V. Solo. The convergence of AML , 1979 .

[6] Dimitri Bertsekas,et al. Distributed dynamic programming , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[7] John N. Tsitsiklis,et al. On the Complexity of Designing Distributed Protocols , 1982, Inf. Control..

[8] Dimitri P. Bertsekas,et al. Distributed asynchronous computation of fixed points , 1983, Math. Program..

[9] John N. Tsitsiklis,et al. Problems in decentralized decision making and computation , 1984 .

[10] P. Kumar,et al. Theory and practice of recursive identification , 1985, IEEE Transactions on Automatic Control.

[11] Dimitri P. Bertsekas,et al. Data Networks , 1986 .

[12] John N. Tsitsiklis,et al. Convergence theories of distributed iterative processes: A survey , 1986 .