Control design for specifications on stochastic hybrid systems

We synthesize controllers for discrete-time stochastic hybrid systems such that the probability of satisfying a given specification on the system is maximized. The specifications are defined with finite state automata. It is shown that automata satisfaction is equivalent to a reachability problem in an extended state space consisting of the system and the automaton state spaces. The control policy is defined as a map from this extended state space to the input space. Using existing results on maximizing reachability probability, the control policy is designed to maximize probability of satisfying the specification.

[1]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[2]  Martin Fränzle,et al.  Satisfaction Meets Expectations - Computing Expected Values of Probabilistic Hybrid Systems with SMT , 2010, IFM.

[3]  W. Rudin.  Principles of mathematical analysis , 1964 .

[4]  Benjamin Van Roy,et al.  The Linear Programming Approach to Approximate Dynamic Programming , 2003, Oper. Res..

[5]  Stephan Merz,et al.  Model Checking , 2000 .

[6]  Onésimo Hernández-Lerma,et al.  Approximation Schemes for Infinite Linear Programs , 1998, SIAM J. Optim..

[7]  C. Belta,et al.  Control of Markov decision processes from PCTL specifications , 2011, Proceedings of the 2011 American Control Conference.

[8]  Rajeev Alur,et al.  A Theory of Timed Automata , 1994, Theor. Comput. Sci..

[9]  Giuseppe Carlo Calafiore,et al.  Uncertain convex programs: randomized solutions and confidence levels , 2005, Math. Program..

[10]  Stephen P. Boyd,et al.  Approximate dynamic programming via iterated Bellman inequalities , 2015 .

[11]  Gerald B. Folland,et al.  Real Analysis: Modern Techniques and Their Applications , 1984 .

[12]  Alessandro Abate,et al.  Adaptive Gridding for Abstraction and Verification of Stochastic Hybrid Systems , 2011, 2011 Eighth International Conference on Quantitative Evaluation of SysTems.

[13]  Lijun Zhang,et al.  Safety Verification for Probabilistic Hybrid Systems , 2010, Eur. J. Control.

[14]  John Lygeros,et al.  Approximate dynamic programming via sum of squares programming , 2012, 2013 European Control Conference (ECC).

[15]  Houssam Abbas,et al.  Linear Hybrid System Falsification through Local Search , 2011, ATVA.

[16]  S. Shankar Sastry,et al.  O-Minimal Hybrid Systems , 2000, Math. Control. Signals Syst..

[17]  John Lygeros,et al.  Verification of discrete time stochastic hybrid systems: A stochastic reach-avoid decision problem , 2010, Autom..

[18]  Christel Baier,et al.  Principles of Model Checking (Representation and Mind Series) , 2008 .

[19]  Christel Baier,et al.  Principles of model checking , 2008 .

[20]  Joost-Pieter Katoen,et al.  Quantitative automata model checking of autonomous stochastic hybrid systems , 2011, HSCC '11.

[21]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[22]  Bengt Jonsson,et al.  A logic for reasoning about time and reliability , 1990, Formal Aspects of Computing.

[23]  Maria Prandini,et al.  Approximate abstractions of stochastic systems: A randomized method , 2011, IEEE Conference on Decision and Control and European Control Conference.

[24]  Calin Belta,et al.  A probabilistic approach for control of a stochastic system from LTL specifications , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[25]  John Lygeros,et al.  Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems , 2008, Autom..

[26]  John Lygeros,et al.  On the connections between PCTL and dynamic programming , 2009, HSCC '10.

[28]  Calin Belta,et al.  A Fully Automated Framework for Control of Linear Systems from Temporal Logic Specifications , 2008, IEEE Transactions on Automatic Control.

[29]  John Lygeros,et al.  Approximate dynamic programming for stochastic reachability , 2013, 2013 European Control Conference (ECC).

[30]  Paulo Tabuada,et al.  Model Checking LTL over Controllable Linear Systems Is Decidable , 2003, HSCC.

[31]  Shobha Venkataraman,et al.  Efficient Solution Algorithms for Factored MDPs , 2003, J. Artif. Intell. Res..