Stochastic control theory and operational research

Abstract The paper gives an overview of stochastic optimal control theory and its applications to operational research. Stochastic control theory deals with the intertemporal optimization of dynamic systems under uncertainty. After a short review of deterministic optimal control theory and the theory of stochastic dynamic systems, a general stochastic optimal control problem is formulated both in discrete and continuous time. Among the method for solving such problems, particularly stochastic dynamic programming and the stochastic maximum principle are introduced. Examples from the areas of finance, advertising, production-inventory models, and information policy illustrate the applicability of stochastic control theory to operational research problems. Moreover, its actual and potential impact on OR is discussed from a methodological point of view.

[1]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Stochastic Control , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[2]  H. Kushner.  On the stochastic maximum principle: Fixed time of control , 1965 .

[3]  K. Hinderer,et al.  Foundations of Non-stationary Dynamic Programming with Discrete Time Parameter , 1970 .

[4]  Charles S. Tapiero,et al.  Stochastic diffusion models with advertising and word-of-mouth effects , 1983 .

[5]  Arthur E. Bryson,et al.  Applied Optimal Control , 1969 .

[6]  Ilkka Virtanen.  Optimal maintenance policy and planned sale date for a machine subject to deterioration and random failure , 1982 .

[7]  Time, dynamics and the process of management modeling , 1978 .

[8]  Suresh P. Sethi,et al.  Simple Models in Stochastic Production Planning. , 1978 .

[9]  Suresh P. Sethi,et al.  Explicit Solution of a General Consumption/Investment Problem , 1986, Math. Oper. Res..

[10]  H. Kushner.  Introduction to stochastic control , 1971 .

[11]  Suresh P. Sethi,et al.  Deterministic and Stochastic Optimization of a Dynamic Advertising Model , 1982 .

[12]  R. G. Vickson,et al.  Optimal Control of Production Sequences: A Continuous Parameter Analysis , 1982, Oper. Res..

[13]  C. Striebel.  Sufficient statistics in the optimum control of stochastic systems , 1965 .

[14]  Gustav Feichtinger,et al.  Anwendungen des Maximumprinzips im Operations Research, Teil 2 , 1982 .

[15]  G. Monahan.  Optimal Advertising with Stochastic Demand , 1983 .

[16]  Y. Bar-Shalom,et al.  Dual effect, certainty equivalence, and separation in stochastic control , 1974 .

[17]  G. Thompson,et al.  Optimal control theory : applications to management science , 1984 .

[18]  G. Siouris,et al.  Optimum systems control , 1979, Proceedings of the IEEE.

[19]  R. C. Merton,et al.  Lifetime Portfolio Selection under Uncertainty: The Continuous-Time Case , 1969 .

[20]  Alain Bensoussan,et al.  Stochastic Production Planning with Production Constraints , 1980 .

[21]  G. Feichtinger.  Anwendungen des Maximumprinzips im Operations Research, Teil 1 , 1982 .

[22]  Cyrus Derman,et al.  Finite State Markovian Decision Processes , 1970 .

[23]  George M. Siouris,et al.  Applied Optimal Control: Optimization, Estimation, and Control , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[24]  A. Jazwinski.  Stochastic Processes and Filtering Theory , 1970 .

[25]  R. F. Hartl.  Eine Übersicht über das Maximumprinzip für optimale Kontrollprobleme mit Zustandsnebenbedingungen , 1984 .

[26]  William A. Brock,et al.  Stochastic methods in economics and finance , 1982 .

[27]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[28]  G. Monahan.  State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 1982 .

[29]  George M. Constantinides,et al.  Existence of Optimal Simple Policies for Discounted-Cost Inventory and Cash Management in Continuous Time , 1978, Oper. Res..

[30]  W. Fleming,et al.  Deterministic and Stochastic Optimal Control , 1975 .

[31]  A. Bensoussan.  Stochastic control by functional analysis methods , 1982 .

[32]  W. D. Ray.  Managerial Planning: An Optimum and Stochastic Control Approach , 1979 .

[33]  Charles S. Tapiero,et al.  On-Line and Adaptive Optimum Advertising Control by a Diffusion Approximation , 1975, Oper. Res..

[34]  S. Kahne,et al.  Optimal control: An introduction to the theory and ITs applications , 1967, IEEE Transactions on Automatic Control.

[35]  L. S. Pontryagin,et al.  Mathematical Theory of Optimal Processes , 1962 .

[36]  Charles S. Tapiero,et al.  Optimum Advertising and Goodwill under Uncertainty , 1978, Oper. Res..

[37]  Onésimo Hernández-Lerma,et al.  Controlled Markov Processes , 1965 .

[38]  Suresh Sethi,et al.  A Survey of Management Science Applications of the Deterministic Maximum Principle , 1978 .

[39]  H. Witsenhausen.  Separation of estimation and control for discrete time systems , 1971 .

[40]  I. Gihman,et al.  Controlled Stochastic Processes , 1979 .

[41]  J. Mossin.  Optimal multiperiod portfolio policies , 1968 .

[42]  Suresh P. Sethi,et al.  Optimal Consumption and Investment Policies Allowing Consumption Constraints and Bankruptcy , 1983, Math. Oper. Res..

[43]  R. Bellman.  Dynamic programming. , 1957, Science.

[44]  D. Naidu,et al.  Optimal Control Systems , 2018 .

[45]  J. Lesourne,et al.  Control theory and the dynamics of the firm , 1982 .

[46]  Chris Beaumont.  Optimal Control Theory and Economic Analysis , 1983 .

[47]  Charles S. Tapiero,et al.  A Mean Variance Approach to the Optimal Machine Maintenance and Replacement Problem , 1979 .

[48]  E. Renshaw,et al.  STOCHASTIC DIFFERENTIAL EQUATIONS , 1974 .

[49]  R. Neck.  Dynamic Systems with Several Decision-Makers , 1982 .

[50]  R. C. Merton,et al.  Optimum consumption and portfolio rules in a continuous - time model Journal of Economic Theory 3 , 1971 .

[51]  J Jaap Wessels,et al.  On using discrete random models within decision support systems , 1984 .

[52]  J. Macedo.  Optimal currency diversification for a class of risk-averse international investors , 1982 .

[53]  K. Åström.  Introduction to Stochastic Control Theory , 1970 .

[54]  Gerald L. Thompson,et al.  Management Applications of Modern Control Theory , 1977 .