Control of Diffusions via Linear Programming

This chapter presents an approach that leverages linear programming to approximate optimal policies for controlled diffusion processes, possibly with high-dimensional state and action spaces. The approach fits a linear combination of basis functions to the dynamic programming value function; the resulting approximation guides control decisions. Linear programming is used here to compute basis function weights. This builds on the linear programming approach to approximate dynamic programming, previously developed in the context of discrete-time stochastic control.

[1]  Tyrone E. Duncan,et al.  Numerical Methods for Stochastic Control Problems in Continuous Time (Harold J. Kushner and Paul G. Dupuis) , 1994, SIAM Rev..

[2]  Claus Munk.  Optimal consumption/investment policies with undiversifiable income risk and liquidity constraints , 2000 .

[3]  H. Kushner.  Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[4]  Alexander C. Wang.  Approximate value iteration approaches to constrained dynamic portfolio problems , 2004 .

[5]  Kenneth L. Judd,et al.  Solving Large Scale Rational Expectations Models , 1997 .

[6]  Benjamin Van Roy,et al.  On Constraint Sampling in the Linear Programming Approach to Approximate Dynamic Programming , 2004, Math. Oper. Res..

[7]  N. Barberis.  Investing for the Long Run When Returns are Predictable , 2000 .

[8]  S. Ross,et al.  A theory of the term structure of interest rates'', Econometrica 53, 385-407 , 1985 .

[9]  Stanley E. Zin,et al.  SPLINE APPROXIMATIONS TO VALUE FUNCTIONS: Linear Programming Approach , 1997 .

[10]  P. Schweitzer,et al.  Generalized polynomial approximations in Markovian decision processes , 1985 .

[11]  Benjamin Van Roy,et al.  The Linear Programming Approach to Approximate Dynamic Programming , 2003, Oper. Res..

[12]  Benjamin Van Roy,et al.  A Cost-Shaping Linear Program for Average-Cost Approximate Dynamic Programming with Performance Guarantees , 2006, Math. Oper. Res..

[13]  Michael W. Brandt.  Estimating Portfolio and Consumption Choice: A Conditional Euler Equations Approach , 1999 .

[14]  A. Lynch,et al.  First Draft : March 1999 This Draft : 25 May 2000 Portfolio Choice and Equity Characteristics : Characterizing the Hedging Demands Induced by Return Predictability , 2000 .

[15]  R. C. Merton,et al.  Optimum consumption and portfolio rules in a continuous - time model Journal of Economic Theory 3 , 1971 .

[16]  N. Krylov.  Controlled Diffusion Processes , 1980 .

[17]  Vivek S. Borkar,et al.  Optimal Control of Diffusion Processes , 1989 .

[18]  D. Duffie.  Dynamic Asset Pricing Theory , 1992 .