Controlled Markov processes and viscosity solutions

This book is intended as an introduction to optimal stochastic control for continuous time Markov processes and to the theory of viscosity solutions. The authors approach stochastic control problems by the method of dynamic programming. The text provides an introduction to dynamic programming for deterministic optimal control problems, as well as to the corresponding theory of viscosity solutions. Also covered are controlled Markov diffusions and viscosity solutions of Hamilton-Jacobi-Bellman equations. The authors have tried, through illustrative examples and selective material, to connect stochastic control theory with other mathematical areas (e.g. large deviations theory) and with applications to engineering, physics, management, and finance.

[1]  R. Courant,et al.  Methods of Mathematical Physics , 1962 .

[2]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1951 .

[3]  Kai Lai Chung,et al.  Markov Chains with Stationary Transition Probabilities , 1961 .

[4]  L. S. Pontryagin,et al.  Mathematical Theory of Optimal Processes , 1962 .

[5]  P. Hartman.  Ordinary Differential Equations , 1965 .

[6]  Leo F. Boron,et al.  Positive solutions of operator equations , 1964 .

[7]  W. Fleming.  Functions of Several Variables , 1965 .

[8]  M. Hestenes.  Calculus of variations and optimal control theory , 1966 .

[9]  R. Bellman.  Dynamic programming. , 1957, Science.

[10]  W. Rudin.  Real and complex analysis , 1968 .

[11]  Herman Chernoff,et al.  Sequential decisions in the control of a spaceship , 1967 .

[12]  O. A. Ladyzhenskai︠a︡,et al.  Linear and quasilinear elliptic equations , 1968 .

[13]  O. Ladyženskaja.  Linear and Quasilinear Equations of Parabolic Type , 1968 .

[14]  Iosif Ilitch Gikhman,et al.  Introduction to the theory of random processes , 1969 .

[15]  A. Friedman.  Differential games , 1971 .

[16]  W. Fleming.  Stochastic Control for Small Noise Intensities , 1971 .

[17]  R. Elliott,et al.  The Existence Of Value In Differential Games , 1972 .

[18]  F. Black,et al.  The Pricing of Options and Corporate Liabilities , 1973, Journal of Political Economy.

[19]  W. Fleming,et al.  Deterministic and Stochastic Optimal Control , 1975 .

[20]  O. J. Oaks,et al.  Piecewise Linear Control of Nonlinear Systems , 1976, IEEE Transactions on Industrial Electronics and Control Instrumentation.

[21]  Mark H. A. Davis.  Linear estimation and stochastic control , 1977 .

[22]  Alʹbert Nikolaevich Shiri︠a︡ev,et al.  Statistics of random processes , 1977 .

[23]  W. Fleming.  Exit probabilities and optimal stochastic control , 1977 .

[24]  Richard B. Vinter,et al.  The Equivalence of Strong and Weak Formulations for Certain Problems in Optimal Control , 1978 .

[25]  Lawrence C. Evans,et al.  A second order elliptic equation with gradient constraint , 1979 .

[26]  J. Aubin.  Mathematical methods of game and economic theory , 1979 .

[27]  D. W. Stroock,et al.  Multidimensional Diffusion Processes , 1979 .

[28]  N. Krylov.  Controlled Diffusion Processes , 1980 .

[29]  On a stochastic control problem with exit constraints , 1980 .

[30]  N. Karoui.  Les Aspects Probabilistes Du Controle Stochastique , 1981 .

[31]  P. Lions.  Control of diffusion processes in RN , 1981 .

[32]  西尾 真喜子.  Lectures on stochastic control theory , 1981 .

[33]  Lawrence C. Evans,et al.  Classical solutions of fully nonlinear, convex, second‐order elliptic equations , 1982 .

[34]  Robert J. Elliott,et al.  Stochastic calculus and applications , 1984, IEEE Transactions on Automatic Control.

[35]  P. Lions.  Generalized Solutions of Hamilton-Jacobi Equations , 1982 .

[36]  L. Rogers.  Stochastic differential equations and diffusion processes: Nobuyuki Ikeda and Shinzo Watanabe North-Holland, Amsterdam, 1981, xiv + 464 pages, Dfl.175.00 , 1982 .

[37]  Maurice Robin,et al.  On some cheap control problems for diffusion processes , 1983 .

[38]  Lamberto Cesari,et al.  Optimization-Theory And Applications , 1983 .

[39]  I. Karatzas.  A class of singular stochastic control problems , 1983, Advances in Applied Probability.

[40]  Ruth J. Williams.  Brownian motion in a wedge with oblique reflection at the boundary , 1985 .

[41]  Lawrence C. Evans,et al.  Classical solutions of the Hamilton-Jacobi-Bellman equation for uniformly elliptic operators , 1983 .

[42]  P. Lions.  Optimal control of diffusion processes and hamilton–jacobi–bellman equations part 2 : viscosity solutions and uniqueness , 1983 .

[43]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[44]  A. Bensoussan,et al.  Optimal control of partially observable stochastic systems with an exponential-of-integral performance index , 1985 .

[45]  Hitoshi Ishii,et al.  Boundary regulatity and uniqueness for an elliptic equations with gradient constraint , 1983 .

[46]  P. Lions.  Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations, Part I , 1983 .

[47]  H. Ishii.  Uniqueness of unbounded viscosity solution of Hamilton-Jacobi equations , 1984 .

[48]  D. Stroock.  An Introduction to the Theory of Large Deviations , 1984 .

[49]  Mark H. A. Davis.  Piecewise‐Deterministic Markov Processes: A General Class of Non‐Diffusion Stochastic Models , 1984 .

[50]  P. Lions,et al.  Some Properties of Viscosity Solutions of Hamilton-Jacobi Equations. , 1984 .

[51]  S. Varadhan.  Large Deviations and Applications , 1984 .

[52]  S. Shreve,et al.  Connections between Optimal Stopping and Singular Stochastic Control I. Monotone Follower Problems , 1984 .

[53]  S. Shreve,et al.  Connections Between Optimal Stopping and Singular Stochastic Control II. Reflected Follower Problems , 1985 .

[54]  I Karatzas,et al.  Probabilistic aspects of finite-fuel stochastic control. , 1985, Proceedings of the National Academy of Sciences of the United States of America.

[55]  R. Akella,et al.  Optimal control of production rate in a failure prone manufacturing system , 1985, 1985 24th IEEE Conference on Decision and Control.

[56]  Hitoshi Ishii,et al.  A PDE approach to some asymptotic problems concerning random differential equations with small noise intensities , 1985 .

[57]  P. Souganidis,et al.  Differential Games, Optimal Control and Directional Derivatives of Viscosity Solutions of Bellman’s and Isaacs’ Equations , 1985 .

[58]  P. Chow,et al.  Additive control of stochastic linear systems with finite horizon , 1985 .

[59]  H. Soner,et al.  On the Singularities of the Viscosity Solutions to Hamilton-Jacobi-Bellman Equations , 1985 .

[60]  S. Sheu.  Stochastic Control and Exit Probabilities of Jump Processes , 1985 .

[61]  B. Øksendal.  Stochastic Differential Equations , 1985 .

[62]  Stanley R. Pliska,et al.  A Stochastic Calculus Model of Continuous Trading: Optimal Portfolios , 1986, Math. Oper. Res..

[63]  U. Haussmann,et al.  A stochastic maximum principle for optimal control of diffusions , 1986 .

[64]  E. Barron,et al.  The Pontryagin maximum principle from dynamic programming and viscosity solutions to first-order partial differential equations , 1986 .

[65]  S. Shreve,et al.  Equivalent models for finite-fuel stochastic control , 1986 .

[66]  H. Soner.  Optimal control with state-space constraint I , 1986 .

[67]  H. Ishii.  Perron’s method for Hamilton-Jacobi equations , 1987 .

[68]  Ioannis Karatzas,et al.  Brownian Motion and Stochastic Calculus , 1987 .

[69]  W. Fleming,et al.  An Optimal Stochastic Production Planning Problem with Randomly Fluctuating Demand , 1987 .

[70]  G. Barles,et al.  Discontinuous solutions of deterministic optimal stopping time problems , 1987 .

[71]  Nicolai V. Krylov,et al.  Nonlinear Elliptic and Parabolic Equations of the Second Order Equations , 1987 .

[72]  A. I. Subbotin,et al.  Game-Theoretical Control Problems , 1987 .

[73]  A. Bensoussan.  Perturbation Methods in Optimal Control , 1988 .

[74]  M. Akian.  Re´solution nume´rique d'equations d'Hamilton-Jacobi-Bellman-au moyen d'algorithmes multigrilles et d'ite´rations sur les politiques , 1988 .

[75]  Michael J. Klass,et al.  A Diffusion Model for Optimal Portfolio Selection in the Presence of Brokerage Fees , 1988, Math. Oper. Res..

[76]  P. Souganidis,et al.  A uniqueness result for viscosity solutions of second order fully nonlinear partial di , 1988 .

[77]  J. Michael Harrison,et al.  Brownian Models of Queueing Networks with Heterogeneous Customer Populations , 1988 .

[78]  K. Glover,et al.  State-space formulae for all stabilizing controllers that satisfy and H ∞ norm bound and relations to risk sensitivity , 1988 .

[79]  W. Fleming,et al.  Convex duality approach to the optimal control of diffusions , 1989 .

[80]  T. Zariphopoulou.  Optimal Investment - Consumption Models With Constraints , 1989 .

[81]  H. Ishii.  On uniqueness and existence of viscosity solutions of fully nonlinear second‐order elliptic PDE's , 1989 .

[82]  Jose-Luis Mendali.  Some estimates for finite difference approximations , 1989 .

[83]  Yun-Gang Chen,et al.  Uniqueness and existence of viscosity solutions of generalized mean curvature flow equations , 1989 .

[84]  F. Clarke.  Methods of dynamic and nonsmooth optimization , 1989 .

[85]  W. Fleming,et al.  Asymptotic expansions for Markov processes with lévy generators , 1989 .

[86]  W. Ziemer.  Weakly differentiable functions , 1989 .

[87]  H. Soner,et al.  Regularity of the value function for a two-dimensional singular stochastic control problem , 1989 .

[88]  A. R. Norman,et al.  Portfolio Selection with Transaction Costs , 1990, Math. Oper. Res..

[89]  Hitoshi Ishii,et al.  The maximum principle for semicontinuous functions , 1990, Differential and Integral Equations.

[90]  P. Whittle.  Risk-Sensitive Optimal Control , 1990 .

[91]  Toshio Mikami,et al.  Variational processes from the weak forward equation , 1990 .

[92]  E. Barron,et al.  Semicontinuous Viscosity Solutions For Hamilton–Jacobi Equations With Convex Hamiltonians , 1990 .

[93]  M. Falcone,et al.  An approximation scheme for the minimum time function , 1990 .

[94]  R. C. Merton,et al.  Continuous-Time Finance , 1990 .

[95]  Eduardo D. Sontag,et al.  Mathematical Control Theory: Deterministic Finite Dimensional Systems , 1990 .

[96]  Suresh P. Sethi,et al.  An Asymptotic Analysis of Hierarchical Control of Manufacturing Systems Under Uncertainty , 1991, Math. Oper. Res..

[97]  M. Bardi,et al.  Hamilton-Jacobi equations with singular boundary conditions on a free boundary and applications to differential games , 1991 .

[98]  S. Sheu.  Some Estimates of the Transition Density of a Nondegenerate Diffusion Markov Process , 1991 .

[99]  H. Soner,et al.  A free boundary problem related to singular stochastic control: the parabolic case , 1991 .

[100]  A. I. Subbotin.  Existence and uniqueness results for Hamilton-Jacobi equations , 1991 .

[101]  Steven E. Shreve,et al.  1optimal Investment and Consumption with Two Bonds and Transaction Costs , 1991 .

[102]  H. Ishii,et al.  Remarks on elliptic singular perturbation problems , 1991 .

[103]  Piermarco Cannarsa,et al.  Some characterizations of optimal trajectories in control theory , 1991 .

[104]  F. Clarke.  Necessary conditions for nonsmooth problems in-optimal control and the calculus of variations , 1991 .

[105]  Piermarco Cannarsa,et al.  Value function and optimality conditions for semilinear control problems , 1992 .

[106]  L. Evans.  Measure theory and fine properties of functions , 1992 .

[107]  Daniel Tataru,et al.  Viscosity solutions of Hamilton-Jacobi equations with unbounded nonlinear terms , 1992 .

[108]  Thaleia Zariphopoulou.  Investment-consumption models with transaction fees and Markov-chain parameters , 1992 .

[109]  H. Soner.  MOTION OF A SET BY THE CURVATURE OF ITS BOUNDARY , 1993 .

[110]  Jakša Cvitanić,et al.  Hedging Contingent Claims with Constrained Portfolios , 1993 .

[111]  H. Frankowska.  Lower semicontinuous solutions of Hamilton-Jacobi-Bellman equations , 1993 .

[112]  H. Ishii.  Viscosity solutions of nonlinear second-order partial differential equations in hilbert spaces , 1993 .

[113]  H. Soner.  Singular perturbations in manufacturing , 1993 .

[114]  Geert Jan Olsder,et al.  Synchronization and Linearity: An Algebra for Discrete Event Systems , 1994 .

[115]  M. James,et al.  Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems , 1994, IEEE Trans. Autom. Control..

[116]  H. Soner,et al.  Optimal Investment and Consumption with Transaction Costs , 1994 .