Optimal strategies for impulse control of piecewise deterministic Markov processes

This paper deals with the general discounted impulse control problem of a piecewise deterministic Markov process. We investigate a new family of epsilon-optimal strategies. The construction of such strategies is explicit and only necessitates the previous knowledge of the cost of the no-impulse strategy. In particular, it does not require the resolution of auxiliary optimal stopping problem or the computation of the value function at each point of the state space. This approach is based on the iteration of a single-jump-or-intervention operator associated to the piecewise deterministic Markov process.

[1]  Dariusz Gatarek,et al.  Optimality conditions for impulsive control of piecewise-deterministic processes , 1992, Math. Control. Signals Syst..

[2]  U. S. Gugerui Optimal stopping of a piecewise-deterministic markov process , 1986 .

[3]  Benoîte de Saporta,et al.  Numerical method for impulse control of piecewise deterministic Markov processes , 2010, Autom..

[4]  Marie Doumic,et al.  Statistical estimation of a growth-fragmentation model observed on a genealogical tree , 2012, 1210.3240.

[5]  Suzanne Lenhart,et al.  Viscosity solutions associated with impulse control problems for piecewise-deterministic processes. , 1989 .

[6]  Jane J. Ye,et al.  Impulse Control of Piecewise Deterministic Markov Processes , 1995 .

[7]  Mark H. A. Davis Piecewise-deterministic Markov processes , 1993 .

[8]  Mark H. Davis Markov Models and Optimization , 1995 .

[9]  Paul Embrechts,et al.  Martingales and insurance risk , 1989 .

[10]  M. H. A. Davis,et al.  Approximations for optimal stopping of a piecewise-deterministic process , 1988, Math. Control. Signals Syst..

[11]  Karen Gonzalez,et al.  Numerical method for optimal stopping of piecewise deterministic Markov processes , 2009, 0903.2114.

[12]  Benoîte de Saporta,et al.  Numerical Methods for Simulation and Optimization of Piecewise Deterministic Markov Processes , 2016 .

[13]  U. Rieder,et al.  Markov Decision Processes with Applications to Finance , 2011 .

[14]  Djalil Chafaï,et al.  On the long time behavior of the TCP window size process , 2008, ArXiv.

[15]  K. Pakdaman,et al.  Fluid limit theorems for stochastic hybrid systems with application to neuron models , 2010, Advances in Applied Probability.

[16]  M. H. A. Davis,et al.  Impulse control of piecewise-deterministic processes , 1989, Math. Control. Signals Syst..