Qualitative analysis of POMDPs with temporal logic specifications for robotics applications

We consider partially observable Markov decision processes (POMDPs), that are a standard framework for robotics applications to model uncertainties present in the real world, with temporal logic specifications. All temporal logic specifications in linear-time temporal logic (LTL) can be expressed as parity objectives. We study the qualitative analysis problem for POMDPs with parity objectives that asks whether there is a controller (policy) to ensure that the objective holds with probability 1 (almost-surely). While the qualitative analysis of POMDPs with parity objectives is undecidable, recent results show that when restricted to finite-memory policies the problem is EXPTIME-complete. While the problem is intractable in theory, we present a practical approach to solve the qualitative analysis problem. We designed several heuristics to deal with the exponential complexity, and have used our implementation on a number of well-known POMDP examples for robotics applications. Our results provide the first practical approach to solve the qualitative analysis of robot motion planning with LTL properties in the presence of uncertainty.

[1]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[2]  Ufuk Topcu,et al.  Abstractions and sensor design in partial-information, reactive controller synthesis , 2014, 2014 American Control Conference.

[3]  David Hsu,et al.  SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces , 2008, Robotics: Science and Systems.

[4]  Blai Bonet,et al.  Solving POMDPs: RTDP-Bel vs. Point-based Algorithms , 2009, IJCAI.

[5]  Krishnendu Chatterjee,et al.  What is Decidable about Partially Observable Markov Decision Processes with omega-Regular Objectives , 2013, CSL.

[6]  Calin Belta,et al.  Control of noisy differential-drive vehicles from time-bounded temporal logic specifications , 2013, 2013 IEEE International Conference on Robotics and Automation.

[7]  Wolfram Burgard,et al.  Coastal navigation-mobile robot navigation with uncertainty in dynamic environments , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[8]  J. Filar,et al.  Competitive Markov Decision Processes , 1996 .

[9]  Krishnendu Chatterjee,et al.  Symbolic algorithms for qualitative analysis of Markov decision processes with Büchi objectives , 2011, Formal Methods Syst. Des..

[10]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[11]  George J. Pappas,et al.  Hybrid Controllers for Path Planning: A Temporal Logic Approach , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[12]  Hadas Kress-Gazit,et al.  Temporal Logic Motion Planning for Mobile Robots , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[13]  Karl Henrik Johansson,et al.  Revising motion planning under Linear Temporal Logic specifications in partially known workspaces , 2013, 2013 IEEE International Conference on Robotics and Automation.

[14]  Reid G. Simmons,et al.  Point-Based POMDP Algorithms: Improved Analysis and Implementation , 2005, UAI.

[15]  Yushan Chen,et al.  LTL robot motion control based on automata learning of environmental dynamics , 2012, 2012 IEEE International Conference on Robotics and Automation.

[16]  Thomas A. Henzinger,et al.  HYTECH: a model checker for hybrid systems , 1997, International Journal on Software Tools for Technology Transfer.

[17]  Leslie Pack Kaelbling,et al.  Grasping POMDPs , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[18]  Lydia E. Kavraki,et al.  Automated model approximation for robotic navigation with POMDPs , 2013, 2013 IEEE International Conference on Robotics and Automation.

[19]  Leslie Pack Kaelbling,et al.  Learning Policies for Partially Observable Environments: Scaling Up , 1997, ICML.

[20]  Nikos A. Vlassis,et al.  A point-based POMDP algorithm for robot planning , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[21]  Hadas Kress-Gazit,et al.  Provably correct continuous control for high-level robot behaviors with actions of arbitrary execution durations , 2013, 2013 IEEE International Conference on Robotics and Automation.

[22]  Hadas Kress-Gazit,et al.  Temporal-Logic-Based Reactive Mission and Motion Planning , 2009, IEEE Transactions on Robotics.

[23]  Christel Baier,et al.  Probabilistic ω-automata , 2012, JACM.

[24]  Lonnie Chrisman,et al.  Reinforcement Learning with Perceptual Aliasing: The Perceptual Distinctions Approach , 1992, AAAI.

[25]  Nir Piterman,et al.  From Nondeterministic Buchi and Streett Automata to Deterministic Parity Automata , 2006, 21st Annual IEEE Symposium on Logic in Computer Science (LICS'06).

[26]  John N. Tsitsiklis,et al.  The Complexity of Markov Decision Processes , 1987, Math. Oper. Res..

[27]  Stephan Merz,et al.  Model Checking , 2000 .

[28]  Reid G. Simmons,et al.  Heuristic Search Value Iteration for POMDPs , 2004, UAI.

[29]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[30]  Mariëlle Stoelinga,et al.  An Introduction to Probabilistic Automata , 2002, Bull. EATCS.

[31]  Krishnendu Chatterjee,et al.  Symbolic Algorithms for Qualitative Analysis of Markov Decision Processes with Büchi Objectives , 2011, CAV.

[32]  Bud Mishra,et al.  Discrete event models+temporal logic=supervisory controller: automatic synthesis of locomotion controllers , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.