Stochastic 2-D Motion Planning with a POMDP Framework

Motion planning is challenging when it comes to the case of imperfect state information. Decision should be made based on belief state which evolves according to the noise from the system dynamics and sensor measurement. In this paper, we propose the QV-Tree Search algorithm which combines the state-of-art offline and online approximation methods for POMDP. Instead of full node expansions in the tree search, only probable future observations are considered through forward sampling. This modification helps reduce online computation time and allows for GPU acceleration. We show using repre- sentative examples that the proposed QV-Tree Search is able to actively localize the robot in order to reach the goal location with high probability. The results of the proposed method is also compared with the A* and MDP algorithms, neither of which handles state uncertainty directly. The comparison shows that QV-Tree Search is able to drive the robot to the goal with higher success rate and fewer steps.

[1]  C. Tomlin,et al.  Closed-loop belief space planning for linear, Gaussian systems , 2011, 2011 IEEE International Conference on Robotics and Automation.

[2]  Milos Hauskrecht,et al.  Value-Function Approximations for Partially Observable Markov Decision Processes , 2000, J. Artif. Intell. Res..

[3]  Reid G. Simmons,et al.  Point-Based POMDP Algorithms: Improved Analysis and Implementation , 2005, UAI.

[4]  Leslie Pack Kaelbling,et al.  Belief space planning assuming maximum likelihood observations , 2010, Robotics: Science and Systems.

[5]  Leslie Pack Kaelbling,et al.  Efficient Planning in Non-Gaussian Belief Spaces and Its Application to Robot Grasping , 2011, ISRR.

[6]  Agostino Martinelli,et al.  Vision and IMU Data Fusion: Closed-Form Solutions for Attitude, Speed, Absolute Scale, and Bias Determination , 2012, IEEE Transactions on Robotics.

[7]  Reid G. Simmons,et al.  Heuristic Search Value Iteration for POMDPs , 2004, UAI.

[8]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[9]  Thierry Siméon,et al.  The Stochastic Motion Roadmap: A Sampling Framework for Planning with Markov Motion Uncertainty , 2007, Robotics: Science and Systems.

[10]  Oussama Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Autonomous Robot Vehicles.

[11]  B. Faverjon,et al.  Probabilistic Roadmaps for Path Planning in High-Dimensional Con(cid:12)guration Spaces , 1996 .

[12]  Emilio Frazzoli,et al.  Sampling-based algorithms for optimal motion planning , 2011, Int. J. Robotics Res..

[13]  Weihong Zhang,et al.  Speeding Up the Convergence of Value Iteration in Partially Observable Markov Decision Processes , 2011, J. Artif. Intell. Res..

[14]  Paolo Valigi,et al.  Exploiting Photometric Information for Planning Under Uncertainty , 2015, ISRR.

[15]  Frank Dellaert,et al.  Planning in the continuous domain: A generalized belief space approach for autonomous navigation in unknown environments , 2015, Int. J. Robotics Res..

[16]  Joelle Pineau,et al.  Online Planning Algorithms for POMDPs , 2008, J. Artif. Intell. Res..

[17]  Brahim Chaib-draa,et al.  AEMS: An Anytime Online Search Algorithm for Approximate Policy Refinement in Large POMDPs , 2007, IJCAI.

[18]  Joelle Pineau,et al.  Point-based value iteration: An anytime algorithm for POMDPs , 2003, IJCAI.

[19]  N. Roy,et al.  The Belief Roadmap: Efficient Planning in Belief Space by Factoring the Covariance , 2009, Int. J. Robotics Res..

[20]  David Hsu,et al.  SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces , 2008, Robotics: Science and Systems.

[21]  Pieter Abbeel,et al.  LQG-MP: Optimized path planning for robots with motion uncertainty and imperfect state information , 2010, Int. J. Robotics Res..

[22]  Nicholas Roy,et al.  Rapidly-exploring Random Belief Trees for motion planning under uncertainty , 2011, 2011 IEEE International Conference on Robotics and Automation.

[23]  Joel Veness,et al.  Monte-Carlo Planning in Large POMDPs , 2010, NIPS.

[24]  Reid G. Simmons,et al.  Particle RRT for Path Planning with Uncertainty , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[25]  RaphaelBertram,et al.  Correction to "A Formal Basis for the Heuristic Determination of Minimum Cost Paths" , 1972 .

[26]  Richard Washington,et al.  BI-POMDP: Bounded, Incremental, Partially-Observable Markov-Model Planning , 1997, ECP.

[27]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[28]  Guy Shani,et al.  Noname manuscript No. (will be inserted by the editor) A Survey of Point-Based POMDP Solvers , 2022 .

[29]  Jaakko Peltonen,et al.  Periodic Finite State Controllers for Efficient POMDP and DEC-POMDP Planning , 2011, NIPS.

[30]  Vijay Kumar,et al.  Robust Stereo Visual Inertial Odometry for Fast Autonomous Flight , 2017, IEEE Robotics and Automation Letters.

[31]  Ji Zhang,et al.  LOAM: Lidar Odometry and Mapping in Real-time , 2014, Robotics: Science and Systems.

[32]  David Hsu,et al.  HyP-DESPOT: A hybrid parallel algorithm for online planning under uncertainty , 2018, Robotics: Science and Systems.

[33]  Roland Siegwart,et al.  Motion‐ and Uncertainty‐aware Path Planning for Micro Aerial Vehicles , 2014, J. Field Robotics.

[34]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[35]  J. Satia,et al.  Markovian Decision Processes with Probabilistic Observation of States , 1973 .

[36]  Sebastian Thrun,et al.  Coastal Navigation with Mobile Robots , 1999, NIPS.

[37]  Vijay Kumar,et al.  Planning Dynamically Feasible Trajectories for Quadrotors Using Safe Flight Corridors in 3-D Complex Environments , 2017, IEEE Robotics and Automation Letters.

[38]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..