Parallel dynamic programming on clusters of workstations

The standard DP (dynamic programming) algorithms are limited by the substantial computational demands they put on contemporary serial computers. In this work, the theory behind the solution to serial monadic dynamic programming problems highlights the theory and application of parallel dynamic programming on a general-purpose architecture (cluster or network of workstations). A simple and well-known technique, message passing, is considered. Several parallel serial monadic DP algorithms are proposed, based on the parallelization in the state variables and the parallelization in the decision variables. Algorithms with no interpolation are also proposed. It is demonstrated how constraints introduce load unbalance which affect scalability and how this problem is inherent to DP.

[1]  Jack Dongarra,et al.  MPI: The Complete Reference , 1996 .

[2]  Robert E. Larson,et al.  State increment dynamic programming , 1968 .

[3]  Fernando Morilla,et al.  Dynamic Programming Predictive Control , 1996 .

[4]  P. Altena,et al.  In search of clusters , 2007 .

[5]  Antonio F. Díaz,et al.  Performance of Message-Passing MATLAB Toolboxes , 2002, VECPAR.

[6]  Shou-Hsuan Stephen Huang,et al.  Parallel Dynamic Programming , 1994, IEEE Trans. Parallel Distributed Syst..

[7]  George Karypis,et al.  Introduction to Parallel Computing Solution Manual , 2003 .

[8]  A. Assad,et al.  Principles of Dynamic Programming, Part II: Advanced Theory and Applications. , 1983 .

[9]  Floyd B. Hanson,et al.  Data parallel solutions of dimensionality problems in stochastic dynamic programming , 1991, [1991] Proceedings of the 30th IEEE Conference on Decision and Control.

[10]  G. Karypis,et al.  Efficient parallel mappings of a dynamic programming algorithm: a summary of results , 1993, [1993] Proceedings Seventh International Parallel Processing Symposium.

[11]  Stuart E. Dreyfus,et al.  Applied Dynamic Programming , 1965 .

[12]  Leon Cooper,et al.  Introduction to Dynamic Programming , 1981 .

[13]  Leopoldo Acosta,et al.  Design of algorithms for spatial-time reduction complexity of dynamic programming , 1992 .

[14]  Sebastián Dormido Canto.  Algoritmo de programación dinámica en paralelo: aplicación a problemas de control , 2001 .

[15]  Vipin Kumar,et al.  Introduction to Parallel Computing , 1994 .

[16]  Sebastián Dormido Bencomo,et al.  Consideraciones sobre la regulación de sistemas mediante técnicas de programación dinámica , 1970 .

[17]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[18]  Benjamin W. Wah,et al.  Multiprocessing of Combinatorial Search Problems , 1985, Computer.

[19]  Carl Kesselman,et al.  Generalized communicators in the Message Passing Interface , 1996, Proceedings. Second MPI Developer's Conference.

[20]  F. Morilla,et al.  Reduction of the dimensionality of dynamic programming: a case study , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[21]  J.-L. Calvet,et al.  Parallelism in dynamic programming: an issue to feedback control , 1992, [Proceedings 1992] The First IEEE Conference on Control Applications.

[22]  John L. Casti,et al.  Principles of Dynamic Programming: Advanced Theory and Applications , 1982 .

[23]  Eric Bach,et al.  Asynchronous Analysis of Parallel Dynamic Programming Algorithms , 1996, IEEE Trans. Parallel Distributed Syst..

[24]  Rajkumar Buyya,et al.  High Performance Cluster Computing , 1999 .

[25]  Robert Edward Larson,et al.  Principles of Dynamic Programming: Basic Analytical and Computational Methods , 1978 .

[26]  Shou-Hsuan Stephen Huang,et al.  Parallel dynamic programming , 1990, Proceedings of the Second IEEE Symposium on Parallel and Distributed Processing 1990.

[27]  Mohan Kumar,et al.  Parallel implementation of the unit commitment problem on NOWs , 1997, Proceedings High Performance Computing on the Information Superhighway. HPC Asia '97.

[28]  Edison Tse,et al.  Parallel Processing Algorithms for the Optimal Control of Nonlinear Dynamic Systems , 1973, IEEE Transactions on Computers.

[29]  K. D. Glazebrook.  Principles of Dynamic Programming, Part 1. Basic Analytic and Computational Methods , 1979 .

[30]  Francisco Almeida,et al.  Paradigms for parallel dynamic programming , 1996, Proceedings of EUROMICRO 96. 22nd Euromicro Conference. Beyond 2000: Hardware and Software Design Strategies.

[31]  D A Pierre,et al.  Optimization Theory with Applications , 1986 .

[32]  Larry Carter,et al.  On the Parallel Execution Time of Tiled Loops , 2003, IEEE Trans. Parallel Distributed Syst..

[33]  E. Mosca.  Optimal, Predictive and Adaptive Control , 1994 .

[34]  Robert E. Larson,et al.  Principles of Dynamic Programming , 1978 .

[35]  Jack Dongarra,et al.  PVM: Parallel virtual machine: a users' guide and tutorial for networked parallel computing , 1995 .

[36]  Michael Allen,et al.  Parallel programming: techniques and applications using networked workstations and parallel computers , 1998 .

[37]  George Karypis,et al.  Introduction to Parallel Computing , 1994 .

[38]  G. Amdhal,et al.  Validity of the single processor approach to achieving large scale computing capabilities , 1967, AFIPS '67 (Spring).

[39]  R. E. Larson,et al.  A dynamic programming successive approximations technique with convergence proofs , 1970 .