Optimal Policy Generation for Partially Satisfiable Co-Safe LTL Specifications

We present a method to calculate cost-optimal policies for task specifications in co-safe linear temporal logic over a Markov decision process model of a stochastic system. Our key contribution is to address scenarios in which the task may not be achievable with probability one. We formalise a task progression metric and, using multi-objective probabilistic model checking, generate policies that are formally guaranteed to, in decreasing order of priority: maximise the probability of finishing the task; maximise progress towards completion, if this is not possible; and minimise the expected time or cost required. We illustrate and evaluate our approach in a robot task planning scenario, where the task is to visit a set of rooms that may be inaccessible during execution.

[1]  Moshe Y. Vardi.  Automatic verification of probabilistic concurrent finite state programs , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[2]  Emilio Frazzoli,et al.  Incremental sampling-based algorithm for minimum-violation motion planning , 2013, 52nd IEEE Conference on Decision and Control.

[3]  Andrey Kolobov,et al.  Saturated Path-Constrained MDP: Planning under Uncertainty and Deterministic Model-Checking Constraints , 2014, AAAI.

[4]  Christel Baier,et al.  Principles of Model Checking (Representation and Mind Series) , 2008 .

[5]  Amir Pnueli.  The Temporal Semantics of Concurrent Programs , 1981, Theor. Comput. Sci..

[6]  Mausam,et al.  A Theory of Goal-Oriented MDPs with Dead Ends , 2012, UAI.

[7]  Calin Belta,et al.  Optimal control of MDPs with temporal logic constraints , 2013, 52nd IEEE Conference on Decision and Control.

[8]  Hadas Kress-Gazit,et al.  Iterative temporal motion planning for hybrid systems in partially unknown environments , 2013, HSCC '13.

[9]  Hongyang Qu,et al.  Quantitative Multi-objective Verification for Probabilistic Systems , 2011, TACAS.

[10]  Christel Baier,et al.  Computing Conditional Probabilities in Markovian Models Efficiently , 2014, TACAS.

[11]  Calin Belta,et al.  Optimal Control of Markov Decision Processes With Linear Temporal Logic Constraints , 2014, IEEE Transactions on Automatic Control.

[12]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[13]  Ufuk Topcu,et al.  Efficient reactive controller synthesis for a fragment of linear temporal logic , 2013, 2013 IEEE International Conference on Robotics and Automation.

[14]  Marta Z. Kwiatkowska,et al.  Automated Verification Techniques for Probabilistic Systems , 2011, SFM.

[15]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[16]  Calin Belta,et al.  Temporal Logic Motion Planning and Control With Probabilistic Satisfaction Guarantees , 2012, IEEE Transactions on Robotics.

[17]  Calin Belta,et al.  Control of noisy differential-drive vehicles from time-bounded temporal logic specifications , 2013, ICRA.

[18]  Nick Hawes,et al.  Optimal and dynamic planning for Markov decision processes with co-safe LTL specifications , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  J. Kemeny,et al.  Denumerable Markov chains , 1969 .

[20]  Xu Chu Ding,et al.  Strategic planning under uncertainties via constrained Markov Decision Processes , 2013, 2013 IEEE International Conference on Robotics and Automation.

[21]  Emilio Frazzoli,et al.  Least-violating control strategy synthesis with safety rules , 2013, HSCC '13.

[22]  Orna Kupferman,et al.  Model Checking of Safety Properties , 1999, CAV.

[23]  Calin Belta,et al.  Incremental control synthesis in probabilistic environments with Temporal Logic constraints , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[24]  Christel Baier,et al.  Principles of model checking , 2008 .

[25]  Florent Teichteil-Königsbuch.  Stochastic Safest and Shortest Path Problems , 2012, AAAI.

[26]  Florent Teichteil-Königsbuch.  Path-Constrained Markov Decision Processes: bridging the gap between probabilistic model-checking and decision-theoretic planning , 2012, ECAI.

[27]  Nick Hawes,et al.  Task scheduling for mobile robots using interval algebra , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).