Control in belief space with Temporal Logic specifications

In this paper, we present a sampling-based algorithm to synthesize control policies with temporal and uncertainty constraints. We introduce a specification language called Gaussian Distribution Temporal Logic (GDTL), an extension of Boolean logic that allows us to incorporate temporal evolution and noise mitigation directly into the task specifications, e.g. “Go to region A and reduce the variance of your state estimate below 0.1 m2.” Our algorithm generates a transition system in the belief space and uses local feedback controllers to break the curse of history associated with belief space planning. Furthermore, conventional automata-based methods become tractable. Switching control policies are then computed using a product Markov Decision Process (MDP) between the transition system and the Rabin automaton encoding the task specification. We present algorithms to translate a GDTL formula to a Rabin automaton and to efficiently construct the product MDP by leveraging recent results from incremental computing. Our approach is evaluated in hardware experiments using a camera network and ground robot.

[1]  C. Tomlin,et al.  Closed-loop belief space planning for linear, Gaussian systems , 2011, 2011 IEEE International Conference on Robotics and Automation.

[2]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[3]  Calin Belta,et al.  Optimality and Robustness in Multi-Robot Path Planning with Temporal Logic Constraints , 2013, Int. J. Robotics Res..

[4]  Aric Hagberg,et al.  Exploring Network Structure, Dynamics, and Function using NetworkX , 2008, Proceedings of the Python in Science Conference.

[5]  Pieter Abbeel,et al.  Scaling up Gaussian Belief Space Planning Through Covariance-Free Trajectory Optimization and Automatic Differentiation , 2014, WAFR.

[6]  N. Roy,et al.  The Belief Roadmap: Efficient Planning in Belief Space by Factoring the Covariance , 2009, Int. J. Robotics Res..

[7]  John N. Tsitsiklis,et al.  The Complexity of Markov Decision Processes , 1987, Math. Oper. Res..

[8]  C. Baier,et al.  Experiments with Deterministic ω-Automata for Formulas of Linear Temporal Logic , 2005 .

[9]  Albert S. Huang,et al.  Estimation, planning, and mapping for autonomous flight using an RGB-D camera in GPS-denied environments , 2012, Int. J. Robotics Res..

[10]  Calin Belta,et al.  Distributed information gathering policies under temporal logic constraints , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[11]  Nicholas Roy,et al.  Rapidly-exploring Random Belief Trees for motion planning under uncertainty , 2011, 2011 IEEE International Conference on Robotics and Automation.

[12]  Calin Belta,et al.  Optimal control of MDPs with temporal logic constraints , 2013, 52nd IEEE Conference on Decision and Control.

[13]  Meeko M. K. Oishi,et al.  Finite state approximation for verification of partially observable stochastic hybrid systems , 2015, HSCC.

[14]  Emilio Frazzoli,et al.  Sampling-based algorithms for optimal motion planning , 2011, Int. J. Robotics Res..

[15]  Emilio Frazzoli,et al.  Sampling-based motion planning with deterministic μ-calculus specifications , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[16]  Pieter Abbeel,et al.  LQG-MP: Optimized path planning for robots with motion uncertainty and imperfect state information , 2010, Int. J. Robotics Res..

[17]  Calin Belta,et al.  Formal Synthesis of Control Policies for Continuous Time Markov Processes From Time-Bounded Temporal Logic Specifications , 2014, IEEE Transactions on Automatic Control.

[18]  Nancy M. Amato,et al.  FIRM: Sampling-based feedback motion-planning under motion uncertainty and imperfect measurements , 2014, Int. J. Robotics Res..

[19]  Joelle Pineau,et al.  Point-based value iteration: An anytime algorithm for POMDPs , 2003, IJCAI.

[20]  Calin Belta,et al.  Distribution temporal logic: Combining correctness with quality of estimation , 2013, 52nd IEEE Conference on Decision and Control.

[21]  Calin Belta,et al.  Reactive sampling-based temporal logic path planning , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Hadas Kress-Gazit,et al.  Iterative temporal motion planning for hybrid systems in partially unknown environments , 2013, HSCC '13.

[23]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[24]  Oliver Brock,et al.  Sampling-Based Motion Planning With Sensing Uncertainty , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[25]  Kris K. Hauser,et al.  Randomized Belief-Space Replanning in Partially-Observable Continuous Spaces , 2010, WAFR.

[26]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision: From Images to Geometric Models , 2003 .

[27]  Wolfram Burgard,et al.  Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) , 2005 .

[28]  Robert E. Tarjan,et al.  Incremental Cycle Detection, Topological Ordering, and Strong Component Maintenance , 2011, ACM Trans. Algorithms.

[29]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[30]  John Lygeros,et al.  Symbolic Control of Stochastic Systems via Approximately Bisimilar Finite Abstractions , 2013, IEEE Transactions on Automatic Control.

[31]  Calin Belta,et al.  Sampling-based temporal logic path planning , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.