Optimal motion planning by reinforcement learning in autonomous mobile vehicles

The aim of this work has been the implementation and testing in real conditions of a new algorithm based on the cell-mapping techniques and reinforcement learning methods to obtain the optimal motion planning of a vehicle considering kinematics, dynamics and obstacle constraints. The algorithm is an extension of the control adjoining cell mapping technique for learning the dynamics of the vehicle instead of using its analytical state equations. It uses a transformation of cell-to-cell mapping in order to reduce the time spent during the learning stage. Real experimental results are reported to show the satisfactory performance of the algorithm.

[1]  Tomás Martínez-Marín,et al.  Motion Planning of a Non-holonomic Vehicle in a Real Environment by Reinforcement Learning* , 2009, IWANN.

[2]  U. Ozguner,et al.  Route planning for OSU-ACT autonomous vehicle in DARPA Urban Challenge , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[3]  Sebastian Thrun,et al.  Stanley: The robot that won the DARPA Grand Challenge , 2006, J. Field Robotics.

[4]  L. Shepp,et al.  OPTIMAL PATHS FOR A CAR THAT GOES BOTH FORWARDS AND BACKWARDS , 1990 .

[5]  Jean-Claude Latombe,et al.  Robot motion planning , 1970, The Kluwer international series in engineering and computer science.

[6]  Andrew W. Moore,et al.  Prioritized Sweeping: Reinforcement Learning with Less Data and Less Time , 1993, Machine Learning.

[7]  Nicholas Roy,et al.  Provably Efficient Learning with Typed Parametric Models , 2009, J. Mach. Learn. Res..

[8]  D. Meziat,et al.  Optimal Control Applied to Wheeled Mobile Vehicles , 2007, 2007 IEEE International Symposium on Intelligent Signal Processing.

[9]  Thierry Fraichard,et al.  Smooth path planning for cars , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[10]  Ho Gi Jung,et al.  Two-touch type parking slot marking recognition for target parking position designation , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[11]  R. Bellman.  Dynamic programming. , 1957, Science.

[12]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[13]  Sujeet Shenoi,et al.  Cell mapping for controller design and evaluation , 1997 .

[14]  Huifang Wang,et al.  A Geometric Algorithm to Compute Time-Optimal Trajectories for a Bidirectional Steered Robot , 2009, IEEE Transactions on Robotics.

[15]  Pascal Morin,et al.  Control of Nonholonomic Mobile Robots Based on the Transverse Function Approach , 2009, IEEE Transactions on Robotics.

[16]  D. Meziat,et al.  Optimal control for Wheeled Mobile Vehicles based on Cell Mapping techniques , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[17]  Klaus-Dieter Kuhnert,et al.  Reinforcement Learning to Drive a Car by Pattern Matching , 2002, DAGM-Symposium.

[18]  Florent Lamiraux,et al.  Smooth motion planning for car-like vehicles , 2001, IEEE Trans. Robotics Autom..

[19]  C. Hsu,et al.  An Unravelling Algorithm for Global Analysis of Dynamical Systems: An Application of Cell-to-Cell Mappings , 1980 .

[20]  P. J. Zufiria,et al.  The adjoining cell mapping and its recursive unraveling, part I: Description of adaptive and recursive algorithms , 1993, Nonlinear Dynamics.

[21]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[22]  P. J. Zufiria,et al.  Improved Optimal Control Methods Based Upon the Adjoining Cell Mapping Technique , 2003 .

[23]  Richard S. Sutton,et al.  Dyna, an integrated architecture for learning, planning, and reacting , 1990, SGAR.

[24]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[25]  Fethi Belkhouche,et al.  Reactive Path Planning in a Dynamic Environment , 2009, IEEE Transactions on Robotics.

[26]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[27]  Andrew W. Moore,et al.  Variable Resolution Discretization in Optimal Control , 2002, Machine Learning.

[28]  U. Ozguner,et al.  A parking algorithm for an autonomous vehicle , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[29]  Tomás Martínez-Marín,et al.  Integration of Cell-Mapping and Reinforcement-Learning Techniques for Motion Planning of Car-Like Robots , 2009, IEEE Transactions on Instrumentation and Measurement.

[30]  C. Hsu.  A discrete method of optimal control based upon the cell state space concept , 1985 .

[31]  H. Kushner.  Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[32]  S. M. Smith,et al.  Cell-state-space-based search , 2002 .

[33]  J. Latombe,et al.  On nonholonomic mobile robots and optimal maneuvering , 1989, Proceedings. IEEE International Symposium on Intelligent Control 1989.