Human-like Highway Trajectory Modeling based on Inverse Reinforcement Learning

Autonomous driving is one of the current cutting edge technologies. For autonomous cars, their driving actions and trajectories should not only achieve autonomy and safety, but also obey human drivers’ behavior patterns, when sharing the roads with other human drivers on the highway. Traditional methods, though robust and interpretable, demands much human labor in engineering the complex mapping from current driving situation to vehicle’s future control. For newly developed deep-learning methods, though they can automatically learn such complex mapping from data and demands fewer humans’ engineering, they mostly act like black-box, and are less interpretable. We proposed a new combined method based on inverse reinforcement learning to harness the advantages of both. Experimental validations on lane-change prediction and human-like trajectory planning show that the proposed method approximates the state-of-the-art performance in modeling human trajectories, and is both interpretable and data-driven.

[1]  Yang Gao,et al.  End-to-End Learning of Driving Models from Large-Scale Video Datasets , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[3]  Emilio Frazzoli,et al.  A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles , 2016, IEEE Transactions on Intelligent Vehicles.

[4]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[5]  Michael Sivak,et al.  A Survey of Public Opinion about Autonomous and Self-Driving Vehicles in the U.S., the U.K., and Australia , 2014 .

[6]  Liran Gispan,et al.  Closing the gap towards end-to-end autonomous vehicle system , 2019, ArXiv.

[7]  Mohan M. Trivedi,et al.  Tactical driver behavior prediction and intent inference: A review , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[8]  Jiebo Luo,et al.  End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perceptions , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[9]  Julius Ziegler,et al.  Optimal trajectory generation for dynamic street scenarios in a Frenét Frame , 2010, 2010 IEEE International Conference on Robotics and Automation.

[10]  David González,et al.  A Review of Motion Planning Techniques for Automated Vehicles , 2016, IEEE Transactions on Intelligent Transportation Systems.

[11]  Yadong Mu,et al.  Learning End-to-End Autonomous Steering Model from Spatial and Temporal Visual Cues , 2017, VSCC '17.

[12]  Markus Wulfmeier,et al.  Watch this: Scalable cost-function learning for path planning in urban environments , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[13]  Hongbin Zha,et al.  Lane change trajectory prediction by using recorded human driving data , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[14]  John F. Canny,et al.  Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[15]  Markus Wulfmeier,et al.  Deep Inverse Reinforcement Learning , 2015, ArXiv.

[16]  Zuduo Zheng,et al.  Recent developments and research needs in modeling lane changing , 2014 .

[17]  Sebastian Scherer,et al.  Integrating kinematics and environment context into deep inverse reinforcement learning for predicting off-road vehicle trajectories , 2018, CoRL.

[18]  Huijing Zhao,et al.  A Human-like Trajectory Planning Method by Learning from Naturalistic Driving Data , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[19]  Josep Perarnau,et al.  Traffic Simulation with Aimsun , 2010 .

[20]  Mykel J. Kochenderfer,et al.  Imitating driver behavior with generative adversarial networks , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[21]  Sergey Levine,et al.  Deep Imitative Models for Flexible Inference, Planning, and Control , 2018, ICLR.

[22]  Jin-Woo Lee,et al.  Automated tactical maneuver discovery, reasoning and trajectory planning for autonomous driving , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[24]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[25]  Mohan M. Trivedi,et al.  Convolutional Social Pooling for Vehicle Trajectory Prediction , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[26]  Catherine M. Burns,et al.  Autonomous Driving in the Real World: Experiences with Tesla Autopilot and Summon , 2016, AutomotiveUI.

[27]  Hongbin Zha,et al.  Learning lane change trajectories from on-road driving data , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[28]  Shuzhi Sam Ge,et al.  Dynamic Motion Planning for Mobile Robots Using Potential Field Method , 2002, Auton. Robots.

[29]  Steven M. LaValle,et al.  RRT-connect: An efficient approach to single-query path planning , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).