Autonomous Planning and Control for Intelligent Vehicles in Traffic

This paper addresses the trajectory planning problem for autonomous vehicles in traffic. We build a stochastic Markov decision process (MDP) model to represent the behaviors of the vehicles. This MDP model takes into account the road geometry and is able to reproduce more diverse driving styles. We introduce a new concept, namely, the “dynamic cell,” to dynamically modify the state of the traffic according to different vehicle velocities, driver intents (signals), and the sizes of the surrounding vehicles (i.e., truck, sedan, and so on). We then use Bézier curves to plan smooth paths for lane switching. The maximum curvature of the path is enforced via certain design parameters. By designing suitable reward functions, different desired driving styles of the intelligent vehicle can be achieved by solving a reinforcement learning problem. The desired driving behaviors (i.e., autonomous highway overtaking) are demonstrated with an in-house developed traffic simulator.

[1]  Hazel Everett,et al.  Interpolation problem with curvature constraints , 2000 .

[2]  Anind K. Dey,et al.  Human Behavior Modeling with Maximum Entropy Inverse Optimal Control , 2009, AAAI Spring Symposium: Human Behavior Modeling.

[3]  Dimitar Filev,et al.  Highway Traffic Modeling and Decision Making for Autonomous Vehicle Using Reinforcement Learning , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[4]  Grzegorz Ślaski,et al.  Method of planning a reference trajectory of a single lane change manoeuver with Bezier curve , 2016 .

[5]  Panagiotis Tsiotras,et al.  Optimal two-point visual driver model and controller development for driver-assist systems for semi-autonomous vehicles , 2016, 2016 American Control Conference (ACC).

[6]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[7]  Tomás Martínez-Marín,et al.  Optimal motion planning by reinforcement learning in autonomous mobile vehicles , 2011, Robotica.

[8]  R. Curry,et al.  Path Planning Based on Bézier Curve for Autonomous Ground Vehicles , 2008, Advances in Electrical and Electronics Engineering - IAENG Special Edition of the World Congress on Engineering and Computer Science 2008.

[9]  Christos Katrakazas,et al.  Real-time motion planning methods for autonomous on-road driving: State-of-the-art and future research directions , 2015 .

[10]  Emilio Frazzoli,et al.  Steady-state cornering equilibria and stabilisation for a vehicle during extreme operating conditions , 2010 .

[11]  Hongliang Yuan,et al.  Autonomous vehicle collision avoidance system using path planning and model-predictive-control-based active front steering and wheel torque control , 2012 .

[12]  R. V. Jategaonkar,et al.  Aerodynamic parameter estimation from flight data applying extended and unscented Kalman filter , 2006 .

[13]  Panagiotis Tsiotras,et al.  Nonlinear Driver Parameter Estimation and Driver Steering Behavior Analysis for ADAS Using Field Test Data , 2017, IEEE Transactions on Human-Machine Systems.

[14]  Rudolph van der Merwe,et al.  The unscented Kalman filter for nonlinear estimation , 2000, Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373).

[15]  Yongjun Zhang,et al.  Road Centerline Extraction in Complex Urban Scenes From LiDAR Data Based on Multiple Features , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Ilya V. Kolmanovsky,et al.  Hierarchical reasoning game theory based approach for evaluation and testing of autonomous vehicle control systems , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[17]  Jonathan P. How,et al.  Real-Time Motion Planning With Applications to Autonomous Urban Driving , 2009, IEEE Transactions on Control Systems Technology.

[18]  Ilya V. Kolmanovsky,et al.  A game theoretical model of traffic with multiple interacting drivers for use in autonomous vehicle development , 2016, 2016 American Control Conference (ACC).

[19]  Gerardo Beruvides,et al.  Obstacle Recognition Based on Machine Learning for On-Chip LiDAR Sensors in a Cyber-Physical System , 2017, Sensors.

[20]  Thierry Fraichard,et al.  From Reeds and Shepp's to continuous-curvature paths , 1999, IEEE Transactions on Robotics.

[21]  Amnon Shashua,et al.  Long-term Planning by Short-term Prediction , 2016, ArXiv.

[22]  Hans B. Pacejka,et al.  Tyre Modelling for Use in Vehicle Dynamics Studies , 1987 .

[23]  Il Hong Suh,et al.  Bezier Curve-Based Smoothing for Path Planner with Curvature Constraint , 2017, 2017 First IEEE International Conference on Robotic Computing (IRC).

[24]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[25]  Sergey Levine,et al.  Continuous Inverse Optimal Control with Locally Optimal Examples , 2012, ICML.

[26]  Masayoshi Tomizuka,et al.  Vehicle Lane Change Maneuver In Automated Highway Systems , 1994 .

[27]  Ahmet Yücesan,et al.  Bézier Curve with a Minimal Jerk Energy , 2016 .

[28]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[29]  Rüdiger Dillmann,et al.  Probabilistic MDP-behavior planning for cars , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[30]  Emilio Frazzoli,et al.  Sampling-based algorithms for optimal motion planning , 2011, Int. J. Robotics Res..

[31]  Tao Mei,et al.  Lane change path planning based on piecewise Bezier curve for autonomous vehicle , 2013, Proceedings of 2013 IEEE International Conference on Vehicular Electronics and Safety.

[32]  Sergey Levine,et al.  Nonlinear Inverse Reinforcement Learning with Gaussian Processes , 2011, NIPS.

[33]  Marcelo H. Ang,et al.  Perception, Planning, Control, and Coordination for Autonomous Vehicles , 2017 .

[34]  B. Francis.  The linear multivariable regulator problem , 1976, 1976 IEEE Conference on Decision and Control including the 15th Symposium on Adaptive Processes.

[35]  Thierry Siméon,et al.  Sampling-Based Path Planning on Configuration-Space Costmaps , 2010, IEEE Transactions on Robotics.

[36]  Emilio Frazzoli,et al.  A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles , 2016, IEEE Transactions on Intelligent Vehicles.

[37]  N. Montés,et al.  Lane changing using s-series clothoidal approximation and dual-rate based on Bezier points to controlling vehicle , 2004 .

[38]  Gabriel Hugh Elkaim,et al.  Contin uous Curvature Path Generation Based on Bezier Curves for Autonomous Vehicles , 2010 .

[39]  Aníbal Ollero,et al.  Dynamic Graph-Search Algorithm for Global Path Planning in Presence of Hazardous Weather , 2013, J. Intell. Robotic Syst..

[40]  Pakharuddin Mohd. Samin,et al.  Integration of Magic Formula Tire Model with Vehicle Handling Model , 2012 .