Distributed Dynamic Programming

with two snitch-over levels for a class of IY/G/I queuing systems Pratin P. Varaiya (!W68-SM78-F'XO), for a photograph and biography. with variable arrival and sewice rate. " Srochasrrc Processes and see this issue. p. 655. At present. he holds the position sity of California. Berkeley. His research interests of Lecturer in the Faculty of Computer Science. Technion-Israel In-are in communication nerworks. stochatic control. and decentralized stitute of Technology. Haifa. s>stems. Ahstruct-We consider distributed algorithms for salting dynamic programming problems whereby several processors participate simultaneously in the computation while maintaining coordination by information ea-change via communication links. A model of asynchronous distributed computation is developed which requires very weak assumptions on the ordering of computations, the timing of information exchange. the amount of local information needed at each computation node. and the initial conditions for the algorithm. The class of problems considered is very broad and includes shortest path problems. and finite and infinite horizon stochastic optimal control problems. When specialized to a shortest path problem the algorithm reduces to the algorithm originall) implemented for routing of messages in the ARPANET. R ECENT advances in microcomputer technology have intensified interest in distributed computation schemes. Aside from modular expandability. other potential advantages of such schemes are a reduction in computation time for solving a given problem due to parallelism of computation. and elimination of the need to communicate problem data available at geographically dispersed data collection points to a computation center. The first advantage is of crucial importance in real-time applications where problem solution time can be an implementation bottleneck. The second advantage manifests itself for example in applications involving communication networks where there is a natural decentralization of problem data acquisition. The structure of dynamic programming naturally lends itself well to distributed computation since it involves calculations that to a great extent can be camed out in parallel. In fact it is trikial to devise simple schemes taking advantage of this structure whereby the calculation involved in each iteration of the standard form of the algorithm is simply shared by several processors. Such schemes require a certain degree of synchronization in that all processors must complete their assigned portion of the computation before a new iteration can begin. As a result complex protocols for algorithm initiation and processor synchronization may be necessary, and the speed of computation is limited to that of the slowest processor. These drawbacks motivate distributed algorithms whereby …