Review of Learning-Based Longitudinal Motion Planning for Autonomous Vehicles: Research Gaps Between Self-Driving and Traffic Congestion

Self-driving technology companies and the research community are accelerating the pace of use of machine learning longitudinal motion planning (mMP) for autonomous vehicles (AVs). This paper reviews the current state of the art in mMP, with an exclusive focus on its impact on traffic congestion. The paper identifies the availability of congestion scenarios in current datasets, and summarizes the required features for training mMP. For learning methods, the major methods in both imitation learning and non-imitation learning are surveyed. The emerging technologies adopted by some leading AV companies, such as Tesla, Waymo, and Comma.ai, are also highlighted. It is found that: (i) the AV industry has been mostly focusing on the long tail problem related to safety and has overlooked the impact on traffic congestion, (ii) the current public self-driving datasets have not included enough congestion scenarios, and mostly lack the necessary input features/output labels to train mMP, and (iii) although the reinforcement learning approach can integrate congestion mitigation into the learning goal, the major mMP method adopted by industry is still behavior cloning, whose capability to learn a congestion-mitigating mMP remains to be seen. Based on the review, the study identifies the research gaps in current mMP development. Some suggestions for congestion mitigation for future mMP studies are proposed: (i) enrich data collection to facilitate the congestion learning, (ii) incorporate non-imitation learning methods to combine traffic efficiency into a safety-oriented technical route, and (iii) integrate domain knowledge from the traditional car-following theory to improve the string stability of mMP.

[1]  G. Moore,et al.  Anomalous collapses of Nares Strait ice arches leads to enhanced export of Arctic sea ice , 2021, Nature communications.

[2]  Dongsuk Kum,et al.  SCALE-Net: Scalable Vehicle Trajectory Prediction Network under Random Number of Interacting Vehicles via Edge-enhanced Graph Convolutional Neural Network , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[3]  Christos Dimitrakakis,et al.  TORCS, The Open Racing Car Simulator , 2005 .

[4]  Luis Miguel Bergasa,et al.  Need data for driver behaviour analysis? Presenting the public UAH-DriveSet , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[5]  Vera Kurková,et al.  Kolmogorov's theorem and multilayer neural networks , 1992, Neural Networks.

[6]  Yang Gao,et al.  Reinforcement Learning from Imperfect Demonstrations , 2018, ICLR.

[7]  Ruigang Yang,et al.  The ApolloScape Dataset for Autonomous Driving , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Mayank Bansal,et al.  ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst , 2018, Robotics: Science and Systems.

[9]  Victor Talpaert,et al.  Exploring applications of deep reinforcement learning for real-world autonomous driving systems , 2019, VISIGRAPP.

[10]  R. Jiang,et al.  Full velocity difference model for a car-following theory. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[11]  Yu-Rui Chen,et al.  End-to-End Deep Learning for Autonomous Longitudinal and Lateral Control based on Vehicle Dynamics , 2018, AIVR 2018.

[12]  Christian Berger,et al.  When to use what data set for your self-driving car algorithm: An overview of publicly available driving datasets , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[13]  Alexandre M. Bayen,et al.  Dissipating stop-and-go waves in closed and open networks via deep reinforcement learning , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[14]  Micol Marchetti-Bowick,et al.  Joint Interaction and Trajectory Prediction for Autonomous Driving using Graph Neural Networks , 2019, ArXiv.

[15]  Srinivas Peeta,et al.  Impact of the Low-level Controller on StringStability of Adaptive Cruise Control System , 2021, ArXiv.

[16]  Xuan Di,et al.  An LSTM-Based Autonomous Driving Model Using Waymo Open Dataset , 2020, ArXiv.

[17]  M Maarten Steinbuch,et al.  Design and implementation of parameterized adaptive cruise control : an explicit model predictive control approach , 2010 .

[18]  Jorge A. Laval,et al.  Car-following behavior characteristics of adaptive cruise control vehicles based on empirical experiments , 2021 .

[19]  Junfeng Zhao,et al.  Virtual Traffic Simulator for Connected and Automated Vehicles , 2019 .

[20]  Christian Laugier,et al.  Comparison of parametric and non-parametric approaches for vehicle speed prediction , 2014, 2014 American Control Conference.

[21]  Biagio Ciuffo,et al.  openACC. An open database of car-following experiments to study the properties of commercial ACC systems , 2020, Transportation Research Part C: Emerging Technologies.

[22]  R. B. Potts,et al.  Car-Following Theory of Steady-State Traffic Flow , 1959 .

[23]  Mykel J. Kochenderfer,et al.  Analysis of microscopic behavior models for probabilistic modeling of driver behavior , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[24]  Maarten Steinbuch,et al.  String-Stable CACC Design and Experimental Validation: A Frequency-Domain Approach , 2010, IEEE Transactions on Vehicular Technology.

[25]  Alessandro De Gloria,et al.  The L3Pilot Data Management Toolchain for a Level 3 Vehicle Automation Pilot , 2020, Electronics.

[26]  R. M. Michaels,et al.  Perceptual Factors in Car-Following , 1963 .

[27]  Simon Lucey,et al.  Argoverse: 3D Tracking and Forecasting With Rich Maps , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Shu Yang,et al.  Baidu driving dataset and end-to-end reactive control model , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[29]  Bernard Ghanem,et al.  Driving Policy Transfer via Modularity and Abstraction , 2018, CoRL.

[30]  Eder Santana,et al.  Exploring the Limitations of Behavior Cloning for Autonomous Driving , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[31]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[32]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[34]  Raia Hadsell,et al.  Learning to Navigate in Cities Without a Map , 2018, NeurIPS.

[35]  Mashrur Chowdhury,et al.  A Review of Communication, Driver Characteristics, and Controls Aspects of Cooperative Adaptive Cruise Control (CACC) , 2016, IEEE Transactions on Intelligent Transportation Systems.

[36]  Nathan van de Wouw,et al.  Design and experimental evaluation of cooperative adaptive cruise control , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[37]  Xiaobo Qu,et al.  A recurrent neural network based microscopic car following model to predict traffic oscillation , 2017 .

[38]  Luke Fletcher,et al.  A perception‐driven autonomous urban vehicle , 2008, J. Field Robotics.

[39]  V. Tikhomirov On the Representation of Continuous Functions of Several Variables as Superpositions of Continuous Functions of one Variable and Addition , 1991 .

[40]  Etienne Perot,et al.  Deep Reinforcement Learning framework for Autonomous Driving , 2017, Autonomous Vehicles and Machines.

[41]  Hema Swetha Koppula,et al.  Brain4Cars: Car That Knows Before You Do via Sensory-Fusion Deep Learning Architecture , 2016, ArXiv.

[42]  Alexander Carballo,et al.  A Survey of Autonomous Driving: Common Practices and Emerging Technologies , 2019, IEEE Access.

[43]  Peter A. Beling,et al.  Graph Convolution Networks for Probabilistic Modeling of Driving Acceleration , 2020, 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC).

[44]  Andreas Geiger,et al.  Conditional Affordance Learning for Driving in Urban Environments , 2018, CoRL.

[45]  Yang Gao,et al.  End-to-End Learning of Driving Models from Large-Scale Video Datasets , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Alireza Talebpour,et al.  Effect of information availability on stability of traffic flow: Percolation theory approach , 2018, Transportation Research Part B: Methodological.

[47]  Eder Santana,et al.  A Commute in Data: The comma2k19 Dataset , 2018, ArXiv.

[48]  Martin Treiber,et al.  The Intelligent Driver Model with Stochasticity -New Insights Into Traffic Flow Oscillations , 2017 .

[49]  Mykel J. Kochenderfer,et al.  Simultaneous policy learning and latent state inference for imitating driver behavior , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[50]  Christopher Goodin,et al.  LiDAR Data Segmentation in Off-Road Environment Using Convolutional Neural Networks (CNN) , 2020 .

[51]  Agr Bullen DEVELOPMENT OF COMPACT MICROSIMULATION FOR ANALYZING FREEWAY OPERATIONS AND DESIGN , 1982 .

[52]  David Janz,et al.  Learning to Drive in a Day , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[53]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[54]  Javier Alonso-Mora,et al.  Planning and Decision-Making for Autonomous Vehicles , 2018, Annu. Rev. Control. Robotics Auton. Syst..

[55]  Germán Ros,et al.  CARLA: An Open Urban Driving Simulator , 2017, CoRL.

[56]  Hesham M. Eraqi,et al.  End-to-End Deep Learning for Steering Autonomous Vehicles Considering Temporal Dependencies , 2017, ArXiv.

[57]  Maria Laura Delle Monache,et al.  Are Commercially Implemented Adaptive Cruise Control Systems String Stable? , 2019, IEEE Transactions on Intelligent Transportation Systems.

[58]  Weidong Cao,et al.  A Survey on Theories and Applications for Self-Driving Cars Based on Deep Learning Methods , 2020, Applied Sciences.

[59]  D. Cremers,et al.  Learning to Drive using Inverse Reinforcement Learning and Deep Q-Networks , 2016, ArXiv.

[60]  Lutz Eckstein,et al.  The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[61]  Steven E Shladover,et al.  Impacts of Cooperative Adaptive Cruise Control on Freeway Traffic Flow , 2012 .

[62]  Rainer Wiedemann,et al.  SIMULATION DES STRASSENVERKEHRSFLUSSES. , 1974 .

[63]  Jian Sun,et al.  Stability analysis methods and their applicability to car-following models in conventional and connected environments , 2018 .

[64]  John M. Dolan,et al.  A robust autonomous freeway driving algorithm , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[65]  Lex Fridman,et al.  DeepTraffic: Crowdsourced Hyperparameter Tuning of Deep Reinforcement Learning Systems for Multi-Agent Dense Traffic Navigation , 2018 .

[66]  Han-Shue Tan,et al.  Design and field testing of a Cooperative Adaptive Cruise Control system , 2010, Proceedings of the 2010 American Control Conference.

[67]  Gaetan Le-Gall,et al.  Imitation Learning for End to End Vehicle Longitudinal Control with Forward Camera , 2018, ArXiv.

[68]  Zhijun Liu,et al.  Hybrid deep reinforcement learning based eco-driving for low-level connected and automated vehicles along signalized corridors , 2021 .

[69]  Masayoshi Tomizuka,et al.  Model-free Deep Reinforcement Learning for Urban Autonomous Driving , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[70]  Emilio Frazzoli,et al.  A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles , 2016, IEEE Transactions on Intelligent Vehicles.

[71]  Srinivas Peeta,et al.  Smooth-Switching Control-Based Cooperative Adaptive Cruise Control by Considering Dynamic Information Flow Topology , 2020 .

[72]  P. G. Gipps,et al.  A behavioural car-following model for computer simulation , 1981 .

[73]  Babak Shahian Jahromi,et al.  Control of autonomous ground vehicles: a brief technical review , 2017 .

[74]  Sergey Levine,et al.  A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models , 2016, ArXiv.

[75]  Hai-Jun Huang,et al.  AN EXTENDED OV MODEL WITH CONSIDERATION OF DRIVER'S MEMORY , 2009 .

[76]  E. Montroll,et al.  Traffic Dynamics: Studies in Car Following , 1958 .

[77]  Xinming Huang,et al.  End-to-end learning for lane keeping of self-driving cars , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[78]  Shobit Sharma,et al.  Lateral and Longitudinal Motion Control of Autonomous Vehicles using Deep Learning , 2019, 2019 IEEE International Conference on Electro Information Technology (EIT).

[79]  Anca D. Dragan,et al.  Planning for Autonomous Cars that Leverage Effects on Human Actions , 2016, Robotics: Science and Systems.

[80]  Xuan Di,et al.  A Survey on Autonomous Vehicle Control in the Era of Mixed-Autonomy: From Physics-Based to AI-Guided Driving Policy Learning , 2020, Transportation Research Part C: Emerging Technologies.

[81]  Stefano Ermon,et al.  Generative Adversarial Imitation Learning , 2016, NIPS.

[82]  Gentry Lee,et al.  A Generalization of Linear Car-Following Theory , 1966, Oper. Res..

[83]  Mohan M. Trivedi,et al.  Convolutional Social Pooling for Vehicle Trajectory Prediction , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[84]  Michael Stolz,et al.  Search-Based Optimal Motion Planning for Automated Driving , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[85]  Christos Katrakazas,et al.  Real-time motion planning methods for autonomous on-road driving: State-of-the-art and future research directions , 2015 .

[86]  Alexey Dosovitskiy,et al.  End-to-End Driving Via Conditional Imitation Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[87]  Bart van Arem,et al.  The Impact of Cooperative Adaptive Cruise Control on Traffic-Flow Characteristics , 2006, IEEE Transactions on Intelligent Transportation Systems.

[88]  Markos Papageorgiou,et al.  Macroscopic traffic flow modeling with adaptive cruise control: Development and numerical solution , 2015, Comput. Math. Appl..

[89]  Nakayama,et al.  Dynamical model of traffic congestion and numerical simulation. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[90]  Mykel J. Kochenderfer,et al.  Multi-Agent Imitation Learning for Driving Simulation , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[91]  John F. Canny,et al.  Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[92]  Qiang Xu,et al.  nuScenes: A Multimodal Dataset for Autonomous Driving , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[93]  Hans-Christian Reuss,et al.  Framework for interactive testing and development of highly automated driving functions , 2018 .

[94]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[95]  Luke Fletcher,et al.  A perception‐driven autonomous urban vehicle , 2008, J. Field Robotics.

[96]  Jorge A. Laval,et al.  Hybrid models of traffic flow : impacts of bounded vehicle accelerations , 2004 .

[97]  Konstantinos Makantasis,et al.  A Deep Reinforcement Learning Driving Policy for Autonomous Road Vehicles , 2019, ArXiv.

[98]  Serge P. Hoogendoorn,et al.  A geometric Brownian motion car-following model: towards a better understanding of capacity drop , 2018, Transportmetrica B: Transport Dynamics.

[99]  Alexandre M. Bayen,et al.  Flow: Architecture and Benchmarking for Reinforcement Learning in Traffic Control , 2017, ArXiv.

[100]  Cewu Lu,et al.  Virtual to Real Reinforcement Learning for Autonomous Driving , 2017, BMVC.

[101]  Maria Laura Delle Monache,et al.  Dissipation of stop-and-go waves via control of autonomous vehicles: Field experiments , 2017, ArXiv.

[102]  Alireza Talebpour,et al.  Influence of connected and autonomous vehicles on traffic flow stability and throughput , 2016 .

[103]  Hani S. Mahmassani,et al.  50th Anniversary Invited Article - Autonomous Vehicles and Connected Vehicle Systems: Flow and Operations Considerations , 2016, Transp. Sci..

[104]  Eric P. Xing,et al.  CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-driving , 2018, ECCV.

[105]  Bowen Tan,et al.  Autonomous Driving in Reality with Reinforcement Learning and Image Translation , 2018, ArXiv.

[106]  Jorge A. Laval,et al.  Analysis of a Two-Regime Stochastic Car-Following Model: Explaining Capacity Drop and Oscillation Instabilities , 2019, Transportation Research Record: Journal of the Transportation Research Board.

[107]  Christian Laugier,et al.  High-speed highway scene prediction based on driver models learned from demonstrations , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[108]  K. Hasebe,et al.  Analysis of optimal velocity model with explicit delay , 1998, patt-sol/9805002.

[109]  Amnon Shashua,et al.  Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving , 2016, ArXiv.

[110]  Jorge A. Laval,et al.  Microscopic modeling of the relaxation phenomenon using a macroscopic lane-changing model , 2008 .

[111]  Mykel J. Kochenderfer,et al.  Imitating driver behavior with generative adversarial networks , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[112]  Dragomir Anguelov,et al.  Scalability in Perception for Autonomous Driving: Waymo Open Dataset , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[113]  Michael J. Pont,et al.  Assessment of high-integrity embedded automotive control systems using hardware in the loop simulation , 2008, J. Syst. Softw..

[114]  Hang Yin,et al.  Test Your Self-Driving Algorithm: An Overview of Publicly Available Driving Datasets and Virtual Testing Environments , 2019, IEEE Transactions on Intelligent Vehicles.

[115]  Paul Newman,et al.  The Oxford Radar RobotCar Dataset: A Radar Extension to the Oxford RobotCar Dataset , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[116]  Dirk Helbing,et al.  Adaptive cruise control design for active congestion avoidance , 2008 .

[117]  Fangyu Wu,et al.  Connections between classical car following models and artificial neural networks , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[118]  Jianxiong Xiao,et al.  DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[119]  Henry X. Liu,et al.  Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment , 2021, Nature Communications.

[120]  Luc Van Gool,et al.  End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners , 2018, ECCV.

[121]  Helbing,et al.  Congested traffic states in empirical observations and microscopic simulations , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[122]  Saber Fallah,et al.  End-to-end Reinforcement Learning for Autonomous Longitudinal Control Using Advantage Actor Critic with Temporal Context , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[123]  Jiebo Luo,et al.  End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perceptions , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[124]  Daniel B. Work,et al.  Model-Based String Stability of Adaptive Cruise Control Systems Using Field Data , 2020, IEEE Transactions on Intelligent Vehicles.

[125]  Hussein Dia,et al.  Neural Agent Car-Following Models , 2007, IEEE Transactions on Intelligent Transportation Systems.

[126]  Jorge A. Laval,et al.  A parsimonious model for the formation of oscillations in car-following models , 2014 .