An adaptive path tracking method for autonomous land vehicle based on neural dynamic programming

Since the nonlinear properties of the autonomous land vehicles (ALVs) and the time-varying relationship between ego-vehicle and the desired path, it is difficult to tune the parameters of a path tracking controller for the autonomous driving of ALVs. Aiming at this problem, a novel learning based path tracking method is proposed in this paper, which is composed of the Stanley control structure and a learning based module. The input of the learning module is the relationship between current vehicle state and the desired path, and the learning output is the parameter k in the Stanley control structure. What we want to learn is to adaptive tune k according to current vehicle state. A near-optimal policy is obtained by neural dynamic programming (NDP), which is an online and model-free algorithm. The learning based module online tunes the parameter k of the Stanley control structure. The simulation results show that the proposed path tracking method possesses attractive performance.

[1]  Peter Vrancx,et al.  Reinforcement Learning: State-of-the-Art , 2012 .

[2]  Francesco Borrelli,et al.  Predictive Active Steering Control for Autonomous Vehicle Systems , 2007, IEEE Transactions on Control Systems Technology.

[3]  Masayoshi Tomizuka,et al.  Optimal Preview Control For Vehicle Lateral Guidance , 1991 .

[4]  Robin S. Sharp.  Driver Steering Control and a New Perspective on Car Handling Qualities , 2005 .

[5]  Ben Tse,et al.  Autonomous Inverted Helicopter Flight via Reinforcement Learning , 2004, ISER.

[6]  Dewen Hu,et al.  Continuous-action reinforcement learning with fast policy search and adaptive basis function selection , 2011, Soft Comput..

[7]  Frank L. Lewis,et al.  Reinforcement learning and optimal adaptive control: An overview and implementation examples , 2012, Annu. Rev. Control..

[8]  C.J. Tomlin,et al.  Autonomous Automobile Trajectory Tracking for Off-Road Driving: Controller Design, Experimental Validation and Racing , 2007, 2007 American Control Conference.

[9]  Sebastian Thrun,et al.  Stanley: The robot that won the DARPA Grand Challenge , 2006, J. Field Robotics.

[10]  Peter Stone,et al.  Reinforcement Learning for RoboCup Soccer Keepaway , 2005, Adapt. Behav..

[11]  Jarrod M. Snider.  Automatic Steering Methods for Autonomous Automobile Path Tracking , 2009 .

[12]  Jennie Si,et al.  Online learning control by association and reinforcement. , 2001, IEEE transactions on neural networks.

[13]  Tsu Chin,et al.  California Partners for Advanced Transit and Highways (PATH) , 2004 .

[14]  C. Samson.  Control of chained systems application to path following and time-varying point-stabilization of mobile robots , 1995, IEEE Trans. Autom. Control..

[15]  Panos M. Pardalos,et al.  Approximate dynamic programming: solving the curses of dimensionality , 2009, Optim. Methods Softw..

[16]  Bart De Schutter,et al.  Reinforcement Learning and Dynamic Programming Using Function Approximators , 2010 .

[17]  Marco Wiering,et al.  Reinforcement Learning , 2014, Adaptation, Learning, and Optimization.

[18]  Daxue Liu,et al.  Ribbon Model based path tracking method for autonomous land vehicle , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  F.L. Lewis,et al.  Reinforcement learning and adaptive dynamic programming for feedback control , 2009, IEEE Circuits and Systems Magazine.

[20]  William Whittaker,et al.  Autonomous driving in urban environments: Boss and the Urban Challenge , 2008, J. Field Robotics.

[21]  Charles E. Thorpe,et al.  Integrated mobile robot control , 1991 .

[22]  Francesco Borrelli,et al.  Robust nonlinear predictive control for semiautonomous ground vehicles , 2014, 2014 American Control Conference.

[23]  Giuseppe Oriolo,et al.  Feedback control of a nonholonomic car-like robot , 1998 .

[24]  Peter I. Corke,et al.  Editorial for issue number 1, Journal of Field Robotics , 2006, J. Field Robotics.