Optimality Issues for a Class of Controlled Singularly Perturbed Stochastic Systems

The present paper aims at studying stochastic singularly perturbed control systems. We begin by recalling the linear (primal and dual) formulations for classical control problems. In this framework, we give necessary and sufficient support criteria for optimality of the measures intervening in these formulations. Motivated by these remarks, in a first step, we provide linearized formulations associated with the value function in the averaged dynamics setting. Second, these formulations are used to infer criteria allowing to identify the optimal trajectory of the averaged stochastic system.

[1]  Götz Grammel.  Singularly perturbed differential inclusions: An averaging approach , 1996 .

[2]  Paola Goatin,et al.  Invariant Sets for Controlled Degenerate Diffusions: A Viscosity Solutions Approach , 1999 .

[3]  N. Krylov.  On the rate of convergence of finite-difference approximations for Bellmans equations with variable coefficients , 2000 .

[4]  V. Veliov.  A generalization of the Tikhonov theorem for singularly perturbed differential inclusions , 1997 .

[5]  D. Goreac.  ON LINEARIZED FORMULATIONS FOR CONTROL PROBLEMS WITH PIECEWISE DETERMINISTIC MARKOV DYNAMICS , 2012 .

[6]  Marc Quincampoix,et al.  Existence of stochastic control under state constraints , 1998 .

[7]  Vivek S. Borkar,et al.  On Existence of Limit Occupational Measures Set of a Controlled Stochastic Differential Equation , 2005, SIAM J. Control. Optim..

[8]  Linear programming solutions of periodic optimization problems: approximation of the optimal control , 2007 .

[9]  D. Goreac,et al.  Mayer and Optimal Stopping Stochastic Control Problems With Discontinuous Cost , 2011 .

[10]  Vivek S. Borkar,et al.  Averaging of Singularly Perturbed Controlled Stochastic Differential Equations , 2007 .

[11]  Richard H. Stockbridge,et al.  On the existence of strict optimal controls for constrained, controlled Markov processes in continuous time , 2012 .

[12]  E. Fridman,et al.  Exact slow-fast decomposition of a class of non-linear singularly perturbed optimal control problems via invariant manifolds , 1999 .

[13]  V. Borkar,et al.  Occupation measures for controlled Markov processes: characterization and optimality , 1996 .

[14]  D. Goreac,et al.  A note on linearization methods and dynamic programming principles for stochastic discontinuous control problems , 2012 .

[15]  Vladimir Gaitsgory,et al.  Multiscale Singularly Perturbed Control Systems: Limit Occupational Measures Sets and Averaging , 2002, SIAM J. Control. Optim..

[16]  M. Quincampoix,et al.  Existence of Asymptotic Values for Nonexpansive Stochastic Control Systems , 2014 .

[17]  Vladimir Gaitsgory,et al.  Linear Programming Approach to Deterministic Infinite Horizon Optimal Control Problems with Discounting , 2009, SIAM J. Control. Optim..

[18]  M. Bardi,et al.  Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations , 1997 .

[19]  V. Gaitsgory.  Suboptimization of singularly perturbed control systems , 1992 .

[20]  A. Leizarowitz,et al.  Limit Occupational Measures Set for a Control System and Averaging of Singularly Perturbed Control Systems , 1999 .

[21]  C. SIAMJ..  LINEAR PROGRAMMING APPROACH TO DETERMINISTIC LONG RUN AVERAGE PROBLEMS OF OPTIMAL CONTROL , 2006 .

[22]  W. Fleming,et al.  Convex duality approach to the optimal control of diffusions , 1989 .

[23]  Desineni S Naidu,et al.  Singular perturbations and time scales in control theory and applications: An overview , 2002 .

[24]  Vladimir Gaitsgory,et al.  On a Representation of the Limit Occupational Measures Set of a Control System with Applications to Singularly Perturbed Control Systems , 2004, SIAM J. Control. Optim..

[25]  M. Quincampoix,et al.  Stochastic Optimal Control and Linear Programming Approach , 2011 .

[26]  D. Goreac,et al.  Linearization Techniques for Controlled Piecewise Deterministic Markov Processes; Application to Zubov’s Method , 2012 .