Multi-objective optimal control of stochastic hybrid systems

We present a dynamic programming based solution to a stochastic optimal control problem for a controlled discrete-time stochastic hybrid system. A general summultiplicative cost function is introduced along with a corresponding dynamic recursion that quantifies the weighted costs of achieving multiple (possibly competing) objectives relevant to the theory of stochastic verification and stochastic model predictive control. It is shown that a valuable collection of multi-objective stochastic optimal control problems can be expressed in the form of the presented general sum-multiplicative cost function. A case study in the early retirement problem (equivalently the reinsurance problem) is presented.

[1]  John Lygeros,et al.  Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems , 2008, Autom..

[2]  John Lygeros,et al.  Verification of discrete time stochastic hybrid systems: A stochastic reach-avoid decision problem , 2010, Autom..

[3]  John Lygeros,et al.  Stochastic Receding Horizon Control With Bounded Control Inputs: A Vector Space Approach , 2009, IEEE Transactions on Automatic Control.

[4]  Joost-Pieter Katoen,et al.  Quantitative automata model checking of autonomous stochastic hybrid systems , 2011, HSCC '11.

[5]  Mark H. Davis.  Markov Models and Optimization , 1995 .

[6]  S. Sastry,et al.  Towars a Theory of Stochastic Hybrid Systems , 2000, HSCC.

[7]  John Lygeros,et al.  On the connections between PCTL and dynamic programming , 2009, HSCC '10.

[8]  John Lygeros,et al.  Reachability Analysis for Controlled Discrete Time Stochastic Hybrid Systems , 2006, HSCC.

[9]  Nicolas Privault.  The Discrete Time Case , 2009 .

[10]  Jostein Paulsen.  Ruin Models with Investment Income , 2010 .

[11]  Jerzy A. Filar,et al.  Stochastic target hitting time and the problem of early retirement , 2004, IEEE Transactions on Automatic Control.

[12]  John Lygeros,et al.  A stochastic reach-avoid problem with random obstacles , 2011, HSCC '11.

[13]  John Lygeros,et al.  Reachability Questions in Piecewise Deterministic Markov Processes , 2003, HSCC.

[14]  John Lygeros,et al.  Towars a Theory of Stochastic Hybrid Systems , 2000, HSCC.

[15]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[16]  M. K. Ghosh,et al.  Ergodic Control of Switching Diffusions , 1997 .