Scaling up Gaussian Belief Space Planning Through Covariance-Free Trajectory Optimization and Automatic Differentiation

Belief space planning provides a principled framework to compute motion plans that explicitly gather information from sensing, as necessary, to reduce uncertainty about the robot and the environment. We consider the problem of planning in Gaussian belief spaces, which are parameterized in terms of mean states and covariances describing the uncertainty. In this work, we show that it is possible to compute locally optimal plans without including the covariance in direct trajectory optimization formulations of the problem. As a result, the dimensionality of the problem scales linearly in the state dimension instead of quadratically, as would be the case if we were to include the covariance in the optimization. We accomplish this by taking advantage of recent advances in numerical optimal control that include automatic differentiation and state of the art convex solvers. We show that the running time of each optimization step of the covariance-free trajectory optimization is \(O(n^3T)\), where \(n\) is the dimension of the state space and \(T\) is the number of time steps in the trajectory. We present experiments in simulation on a variety of planning problems under uncertainty including manipulator planning, estimating unknown model parameters for dynamical systems, and active simultaneous localization and mapping (active SLAM). Our experiments suggest that our method can solve planning problems in \(100\) dimensional state spaces and obtain computational speedups of \(400\times \) over related trajectory optimization methods .

[1]  John T. Betts,et al.  Practical Methods for Optimal Control and Estimation Using Nonlinear Programming , 2009 .

[2]  P. Abbeel,et al.  LQG-MP: Optimized path planning for robots with motion uncertainty and imperfect state information , 2011 .

[3]  Nicholas Roy,et al.  Global Motion Planning under Uncertain Motion, Sensing, and Environment Map , 2012 .

[4]  Gamini Dissanayake,et al.  Planning under uncertainty using model predictive control for information gathering , 2006, Robotics Auton. Syst..

[5]  Moritz Diehl,et al.  CasADi -- A symbolic package for automatic differentiation and optimal control , 2012 .

[6]  Ron Alterovitz,et al.  Motion planning under uncertainty using iterative local optimization in belief space , 2012, Int. J. Robotics Res..

[7]  C. Tomlin,et al.  Closed-loop belief space planning for linear, Gaussian systems , 2011, 2011 IEEE International Conference on Robotics and Automation.

[8]  Pieter Abbeel,et al.  Active exploration using trajectory optimization for robotic grasping in the presence of occlusions , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[9]  David Hsu,et al.  Integrated perception and planning in the continuous space: A POMDP approach , 2013, Int. J. Robotics Res..

[10]  William D. Smart,et al.  A Scalable Method for Solving High-Dimensional Continuous POMDPs Using Local Approximation , 2010, UAI.

[11]  N. Roy,et al.  The Belief Roadmap: Efficient Planning in Belief Space by Factoring the Covariance , 2009, Int. J. Robotics Res..

[12]  Max Donath,et al.  American Control Conference , 1993 .

[13]  Eduardo F. Camacho,et al.  Model Predictive Controllers , 2007 .

[14]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[15]  Manfred Morari,et al.  Efficient interior point methods for multistage problems arising in receding horizon control , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[16]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[17]  Pieter Abbeel,et al.  Finding Locally Optimal, Collision-Free Trajectories with Sequential Convex Optimization , 2013, Robotics: Science and Systems.

[18]  Wolfram Burgard,et al.  Information Gain-based Exploration Using Rao-Blackwellized Particle Filters , 2005, Robotics: Science and Systems.

[19]  Kris K. Hauser,et al.  Randomized Belief-Space Replanning in Partially-Observable Continuous Spaces , 2010, WAFR.

[20]  Nicholas Roy,et al.  Rapidly-exploring Random Belief Trees for motion planning under uncertainty , 2011, 2011 IEEE International Conference on Robotics and Automation.

[21]  Juan Andrade-Cetto,et al.  Planning Reliable Paths With Pose SLAM , 2013, IEEE Transactions on Robotics.

[22]  Geoffrey A. Hollinger,et al.  Sampling-based Motion Planning for Robotic Information Gathering , 2013, Robotics: Science and Systems.

[23]  Evangelos Theodorou,et al.  Multi-robot active SLAM with relative entropy optimization , 2013, 2013 American Control Conference.

[24]  Ron Alterovitz,et al.  Efficient Approximate Value Iteration for Continuous Gaussian POMDPs , 2012, AAAI.

[25]  Jan Peters,et al.  Solving Nonlinear Continuous State-Action-Observation POMDPs for Mechanical Systems with Gaussian Noise , 2012, EWRL 2012.

[26]  Jur P. van den Berg,et al.  Online parameter estimation via real-time replanning of continuous Gaussian POMDPs , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[27]  John N. Tsitsiklis,et al.  The Complexity of Markov Decision Processes , 1987, Math. Oper. Res..

[28]  Leslie Pack Kaelbling,et al.  Belief space planning assuming maximum likelihood observations , 2010, Robotics: Science and Systems.

[29]  Frank Dellaert,et al.  Towards Planning in Generalized Belief Space , 2013, ISRR.

[30]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[31]  Pieter Abbeel,et al.  Autonomous multilateral debridement with the Raven surgical robot , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[32]  Pieter Abbeel,et al.  Gaussian belief space planning with discontinuities in sensing domains , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[33]  Brahim Chaib-draa,et al.  Bayesian reinforcement learning in continuous POMDPs with gaussian processes , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[34]  Hannes Sommer,et al.  Automatic Differentiation on Differentiable Manifolds as a Tool for Robotics , 2013, ISRR.

[35]  Pascal Poupart,et al.  Point-Based Value Iteration for Continuous POMDPs , 2006, J. Mach. Learn. Res..

[36]  Geoffrey A. Hollinger,et al.  Stochastic Motion Planning for Robotic Information Gathering , 2013 .

[37]  Razvan Pascanu,et al.  Theano: A CPU and GPU Math Compiler in Python , 2010, SciPy.

[38]  Leslie Pack Kaelbling,et al.  Integrated task and motion planning in belief space , 2013, Int. J. Robotics Res..

[39]  Nancy M. Amato,et al.  FIRM: Sampling-based feedback motion-planning under motion uncertainty and imperfect measurements , 2014, Int. J. Robotics Res..

[40]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[41]  Anil V. Rao,et al.  Practical Methods for Optimal Control Using Nonlinear Programming , 1987 .