Asymptotic Control for a Class of Piecewise Deterministic Markov Processes Associated to Temperate Viruses

We aim at characterizing the asymptotic behavior of value functions in the control of piece-wise deterministic Markov processes (PDMP) of switch type under nonexpansive assumptions. For a particular class of processes inspired by temperate viruses, we show that uniform limits of discounted problems as the discount decreases to zero and time-averaged problems as the time horizon increases to infinity exist and coincide. The arguments allow the limit value to depend on initial configuration of the system and do not require dissipative properties on the dynamics. The approach strongly relies on viscosity techniques, linear programming arguments and coupling via random measures associated to PDMP. As an intermediate step in our approach, we present the approximation of discounted value functions when using piecewise constant (in time) open-loop policies.

[1]  Manfred Schäl,et al.  Piecewise Deterministic Markov Control Processes with Feedback Controls and Unbounded Costs , 2004 .

[2]  J. Aubin.  Set-valued analysis , 1990 .

[3]  Jérôme Renault.  Uniform value in Dynamic Programming , 2008, 0803.2758.

[4]  Mark H. A. Davis.  Piecewise‐Deterministic Markov Processes: A General Class of Non‐Diffusion Stochastic Models , 1984 .

[5]  M. Arisawa.  Ergodic problem for the Hamilton-Jacobi-Bellman equation. II , 1998 .

[6]  P. Lions,et al.  ON ERGODIC STOCHASTIC CONTROL , 1998 .

[7]  Vivek S. Borkar,et al.  Averaging of Singularly Perturbed Controlled Stochastic Differential Equations , 2007 .

[8]  Khashayar Pakdaman,et al.  Intrinsic variability of latency to first-spike , 2010, Biol. Cybern..

[9]  P. K. Pollett,et al.  POINT PROCESSES AND QUEUES Martingale Dynamics (Springer Series in Statistics) , 1984 .

[10]  Miquel Oliu-Barton,et al.  A uniform Tauberian theorem in optimal control , 2010, 1004.4174.

[11]  Adrien Richou,et al.  Ergodic BSDEs and related PDEs with Neumann boundary conditions , 2008, 0807.1521.

[12]  J. Hasty,et al.  Noise-based switches and amplifiers for gene expression. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[13]  M. Benaim,et al.  Quantitative ergodicity for some switched dynamical systems , 2012, 1204.1922.

[14]  Mark H. A. Davis.  Control of piecewise-deterministic processes via discrete-time dynamic programming , 1986 .

[15]  M. Quincampoix,et al.  Existence of Asymptotic Values for Nonexpansive Stochastic Control Systems , 2014 .

[16]  D. Goreac,et al.  Uniform Assymptotics in the Average Continuous Control of Piecewise Deterministic Markov Processes : Vanishing Approach , 2013 .

[17]  M. Dempster,et al.  Generalized Bellman-Hamilton-Jacobi optimality conditions for a control problem with a boundary condition , 1996 .

[18]  D. Goreac,et al.  Viability, invariance and reachability for controlled piecewise deterministic Markov processes associated to gene networks , 2010, 1002.2242.

[19]  Carl Graham,et al.  Interacting multi-class transmissions in large stochastic networks , 2008, 0810.0347.

[20]  Gopal K. Basak,et al.  Ergodic control of degenerate diffusions , 1997 .

[21]  Tomasz Rolski,et al.  Stochastic Processes for Insurance and Finance , 2001 .

[22]  F. Dufour,et al.  Continuous Control of Piecewise Deterministic Markov Processes with Long Run Average Cost , 2011 .

[23]  Marc Quincampoix,et al.  On the Existence of a Limit Value in Some Nonexpansive Optimal Control Problems , 2009, SIAM J. Control. Optim..

[24]  L. Rogers.  Stochastic differential equations and diffusion processes: Nobuyuki Ikeda and Shinzo Watanabe North-Holland, Amsterdam, 1981, xiv + 464 pages, Dfl.175.00 , 1982 .

[26]  Wainrib Gilles,et al.  Intrinsic variability of latency to first-spike , 2010 .

[27]  Oswaldo Luiz do Valle Costa,et al.  Average Impulse Control of Piecewise Deterministic Processes , 1989 .

[28]  N. Krylov.  On the rate of convergence of finite-difference approximations for Bellmans equations with variable coefficients , 2000 .

[29]  Dariusz Gatarek,et al.  Optimality conditions for impulsive control of piecewise-deterministic processes , 1992, Math. Control. Signals Syst..

[30]  Piernicola Bettiol,et al.  On ergodic problem for Hamilton-Jacobi-Isaacs equations , 2005 .

[31]  H Zhang,et al.  Piecewise deterministic Markov processes and dynamic reliability , 2006 .

[32]  Onno Boxma,et al.  ON/OFF STORAGE SYSTEMS WITH STATE-DEPENDENT INPUT, OUTPUT, AND SWITCHING RATES , 2005, Probability in the Engineering and Informational Sciences.

[33]  P. Brémaud.  Point processes and queues, martingale dynamics , 1983 .

[34]  Oswaldo Luiz do Valle Costa,et al.  Continuous Average Control of Piecewise Deterministic Markov Processes , 2013 .

[35]  Rainer Buckdahn,et al.  Limit Theorem for Controlled Backward SDEs and Homogenization of Hamilton–Jacobi–Bellman Equations , 2005 .

[36]  Heinz Schättler,et al.  Optimal Control with State Space Constraints , 2015, Encyclopedia of Systems and Control.

[37]  Ovidiu Radulescu,et al.  Convergence of stochastic gene networks to hybrid piecewise deterministic processes , 2011, 1101.1431.

[38]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[39]  D. Goreac,et al.  Linearization Techniques for Controlled Piecewise Deterministic Markov Processes; Application to Zubov’s Method , 2012 .

[40]  Ovidiu Radulescu,et al.  Hybrid stochastic simplifications for multiscale gene networks , 2009, BMC Systems Biology.

[41]  Z. Artstein,et al.  The Value Function of Singularly Perturbed Control Systems , 2000 .

[42]  G. Kallianpur.  Stochastic differential equations and diffusion processes , 1981 .

[43]  F. Dufour,et al.  The Vanishing Discount Approach for the Average Continuous Control of Piecewise Deterministic Markov Processes , 2009, Journal of Applied Probability.

[44]  ANTHONY ALMUDEVAR,et al.  A Dynamic Programming Algorithm for the Optimal Control of Piecewise Deterministic Markov Processes , 2001, SIAM J. Control. Optim..

[45]  S. Tapscott,et al.  Modeling stochastic gene expression: implications for haploinsufficiency. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[46]  G. H. Hardy,et al.  Tauberian Theorems Concerning Power Series and Dirichlet's Series whose Coefficients are Positive* , 1914 .

[47]  Mark H. Davis.  Markov Models and Optimization , 1995 .

[48]  Guy Barles,et al.  On the convergence rate of approximation schemes for Hamilton-Jacobi-Bellman equations , 2002 .

[49]  N. Krylov,et al.  Approximating Value Functions for Controlled Degenerate Diffusion Processes by Using Piece-Wise Constant Policies , 1999 .

[50]  T. Rolski.  Stochastic Processes for Insurance and Finance , 1999 .

[51]  M. Jacobsen.  Point Process Theory and Applications: Marked Point and Piecewise Deterministic Processes , 2005 .