Enforcing temporal logic specifications via reinforcement learning

We consider the problem of controlling a system with unknown, stochastic dynamics to achieve a complex, time-sensitive task. An example of this problem is controlling a noisy aerial vehicle with partially known dynamics to visit a pre-specified set of regions in any order while avoiding hazardous areas. In particular, we are interested in tasks which can be described by signal temporal logic (STL) specifications. STL is a rich logic that can be used to describe tasks involving bounds on physical parameters, continuous time bounds, and logical relationships over time and states. STL is equipped with a continuous measure called the robustness degree that measures how strongly a given sample path exhibits an STL property [4, 3]. This measure enables the use of continuous optimization problems to solve learning [7, 6] or formal synthesis problems [9] involving STL.

[1]  Calin Belta,et al.  Temporal logic inference for classification and prediction from data , 2014, HSCC.

[2]  L. M. Bujorianu,et al.  Approximate Abstractions of Stochastic Hybrid Systems , 2008 .

[3]  Vasumathi Raman,et al.  Model predictive control from signal temporal logic specifications: a case study , 2014, CyPhy '14.

[4]  George J. Pappas,et al.  Robustness of temporal logic specifications for continuous-time signals , 2009, Theor. Comput. Sci..

[5]  Alberto L. Sangiovanni-Vincentelli,et al.  Model predictive control with signal temporal logic specifications , 2014, 53rd IEEE Conference on Decision and Control.

[6]  S. Shankar Sastry,et al.  A learning based approach to control synthesis of Markov decision processes for linear temporal logic specifications , 2014, 53rd IEEE Conference on Decision and Control.

[7]  John N. Tsitsiklis,et al.  Asynchronous Stochastic Approximation and Q-Learning , 1994, Machine Learning.

[8]  Calin Belta,et al.  Anomaly detection in cyber-physical systems: A formal methods approach , 2014, 53rd IEEE Conference on Decision and Control.

[9]  Georgios E. Fainekos,et al.  On-Line Monitoring for Temporal Logic Robustness , 2014, RV.

[10]  Calin Belta,et al.  Approximate Markovian abstractions for linear stochastic systems , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[11]  Ufuk Topcu,et al.  Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraints , 2014, Robotics: Science and Systems.

[12]  Oded Maler,et al.  Robust Satisfaction of Temporal Logic over Real-Valued Signals , 2010, FORMATS.