Markov chain approximation methods on generalized HJB equation

This work is concerned with numerical methods for a class of stochastic control optimizations and stochastic differential games. Numerical procedures based on Markov chain approximation techniques are developed in a framework of generalized Hamilton-Jacobi-Bellman equations. Convergence of the algorithms is derived by means of viscosity solution methods.

[1]  G. Yin,et al.  Discrete-Time Markov Chains: Two-Time-Scale Methods and Applications , 2004 .

[2]  P. Souganidis.  Two-Player, Zero-Sum Differential Games and Viscosity Solutions , 1999 .

[3]  X. S. Lin.  Stochastic Processes for Insurance and Finance. By T. Rolski, H. Schmidli, V. Schmidt and J. Teugels (John Wiley, Chichester, 1999) , 2000, British Actuarial Journal.

[4]  H. Kushner.  Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[5]  Gang George Yin,et al.  Numerical methods for controlled regime-switching diffusions and regime-switching jump diffusions , 2006, Autom..

[6]  Qing Zhang,et al.  Continuous-Time Markov Chains and Applications , 1998 .

[7]  Paul Embrechts,et al.  Stochastic processes in insurance and finance , 2001 .

[8]  P. Lions,et al.  User’s guide to viscosity solutions of second order partial differential equations , 1992, math/9207212.

[9]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[10]  Tamer Basar,et al.  H∞-Optimal Control and Related , 1991 .

[11]  Csaba Szepesvári,et al.  A Generalized Reinforcement-Learning Model: Convergence and Applications , 1996, ICML.

[12]  J. Quadrat.  Numerical methods for stochastic control problems in continuous time , 1994 .

[13]  Dimitri P. Bertsekas,et al.  Dynamic Programming: Deterministic and Stochastic Models , 1987 .

[14]  Wolfgang J. Runggaldier,et al.  Mean-variance hedging of options on stocks with Markov volatilities , 1995 .

[15]  HAROLD J. KUSHNER,et al.  Numerical Approximations for Stochastic Differential Games , 2002, SIAM J. Control. Optim..

[16]  G. Yin,et al.  Existence of Saddle Points in Discrete Markov Games and Its Application in Numerical Methods for Stochastic Differential Games , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[17]  Q. Zhang,et al.  Stock Trading: An Optimal Selling Rule , 2001, SIAM J. Control. Optim..

[18]  G. Barles,et al.  Convergence of approximation schemes for fully nonlinear second order equations , 1991 .

[19]  Andrew L. Rukhin,et al.  Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach , 2001, Technometrics.

[20]  R. Elliott,et al.  The Existence Of Value In Differential Games , 1972 .