Stochastic Control with Imperfect Models

We consider the problem of worst case performance estimation for a stochastic dynamic model in the presence of model uncertainty. This is cast as a nonclassical controlled diffusion problem. An infinite dimensional linear programming formulation is given and its dual is derived. The dual is successively approximated on a bounded domain by a semi-infinite and a finite linear program. This uses function approximation based on a reproducing kernel Hilbert space. Error analysis for the approximation is provided along with an estimate of the sample complexity.

[1]  S. Smale,et al.  Shannon sampling II: Connections to learning theory , 2005 .

[2]  R. Stockbridge.  Time-Average Control of Martingale Problems: A Linear Programming Formulation , 1990 .

[3]  M. K. Ghosh,et al.  Controlled diffusions with constraints , 1990 .

[4]  Ding-Xuan Zhou,et al.  The covering number in learning theory , 2002, J. Complex..

[5]  W. Fleming.  Book Review: Discrete-time Markov control processes: Basic optimality criteria , 1997 .

[6]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[7]  V. Borkar.  Probability Theory: An Advanced Course , 1995 .

[8]  R. Stockbridge,et al.  Numerical Comparison of Controls and Verification of Optimality for Stochastic Control Problems , 2000 .

[9]  David Lando,et al.  Credit Risk Modeling , 2009 .

[10]  C. Villani.  Topics in Optimal Transportation , 2003 .

[11]  S. Smale,et al.  ESTIMATING THE APPROXIMATION ERROR IN LEARNING THEORY , 2003 .

[12]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[13]  Onésimo Hernández-Lerma,et al.  The Linear Programming Approach , 2002 .

[14]  Gautam Appa,et al.  Linear Programming in Infinite-Dimensional Spaces , 1989 .

[15]  S. Smale,et al.  Shannon sampling and function reconstruction from point values , 2004 .

[16]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[17]  M. Fowler,et al.  Function Spaces , 2022 .

[18]  John S. Edwards,et al.  Linear Programming and Finite Markovian Control Problems , 1983 .

[19]  V. Borkar,et al.  Occupation measures for controlled Markov processes: characterization and optimality , 1996 .

[20]  Andrew J. Heunis.  On the Prevalence of Stochastic Differential Equations with Unique Strong Solutions , 1986 .

[21]  Vivek S. Borkar,et al.  Optimal Control of Diffusion Processes , 1989 .

[22]  R. Stockbridge,et al.  Approximation of Infinite-Dimensional Linear Programming Problems which Arise in Stochastic Control , 1998 .

[23]  O. Hernández-Lerma,et al.  Further topics on discrete-time Markov control processes , 1999 .

[24]  Onésimo Hernández-Lerma,et al.  Approximation Schemes for Infinite Linear Programs , 1998, SIAM J. Optim..

[25]  O. Hernández-Lerma,et al.  Linear Programming Approximations for Markov Control Processes in Metric Spaces , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[26]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[27]  Vivek S. Borkar,et al.  Convex Analytic Methods in Markov Decision Processes , 2002 .

[28]  C. SIAMJ..  LINEAR PROGRAMMING APPROACH TO DETERMINISTIC LONG RUN AVERAGE PROBLEMS OF OPTIMAL CONTROL , 2006 .