Approximate dynamic programming: solving the curses of dimensionality

Approximate dynamic programming: solving the curses of dimensionality, by Warren B. Powell, Wiley Series in Probability and Statistics, Hoboken, NJ, J. Wiley & Sons, 2007, 488 pp., US$83.60 (hardcover), ISBN 978-0-470-17155-4 Dynamic programming introduced by Bellman back in the 1950s offers a unified approach to solving problems arising in various applications, such as stochastic control or managing entire economies. The principles of dynamic programming have also been widely used in solving largescale stochastic reservoir models. However, due to the three curses of dimensionality, the sizes of a state space as well as the size of the outcome and action space typically grow exponentially in the number of state variables. Therefore, dynamic programming has limited practical applicability when used to find the exact solution of large-scale problems. One approach dealing with the curses of dimensionality is approximate dynamic programming. This book provides a unified and insightful treatment of approximate dynamic programming by integrating four distinct disciplines: Markov design processes, optimization, simulation, and statistics. Approximate Dynamic Programming is a comprehensive book that is designed to offer an introduction to the field. The book provides detailed coverage of modelling decision processes under uncertainty, robustness, designing and estimating value function approximations, choosing effective step-size rules, and convergence issues. In summary, the book Approximate Dynamic Programming is well written, highly pedagogical, with a clear and precise presentation of the material. Each chapter ends with bibliographic notes and well-selected problems that make this material an excellent textbook for advanced undergraduate and beginning graduate students. In addition, the author offers a companion webpage that includes additional problems and their solutions, as well as data sets to reinforce the book’s main concepts. I highly recommend this book for everyone interested in learning the theory and applications of approximate dynamic programming.