MATHEMATICAL ISSUES IN DYNAMIC PROGRAMMING

A general theory of dynamic programming must deal with the formidable mathematical questions that arise from the presence of uncountable probability spaces. These questions are explored at some length by means of a simple example in Section 2. With this example as motivation, the mathematical preliminaries necessary for the construction of a general finite horizon model are developed in Section 3. In Section 4, the results of Section 3 are applied to set up the model and to indicate how a valid dynamic programming algorithm can be defined. The purpose of the paper is to provide some orientation for the development of a comprehensive and mathematically rigorous theory of dynamic programming, as given in the authors’ book “Stochastic Optimal Control: The Discrete-Time Case,” Academic Press, 1978 (republished by Athena Scientific, 1996). This book contains a detailed analysis of finite and infinite horizon problems, and provides references to earlier research.