A Survey on Autonomous Vehicle Control in the Era of Mixed-Autonomy: From Physics-Based to AI-Guided Driving Policy Learning

This paper serves as an introduction and overview of the potentially useful models and methodologies from artificial intelligence (AI) into the field of transportation engineering for autonomous vehicle (AV) control in the era of mixed autonomy. We will discuss state-of-the-art applications of AI-guided methods, identify opportunities and obstacles, raise open questions, and help suggest the building blocks and areas where AI could play a role in mixed autonomy. We divide the stage of autonomous vehicle (AV) deployment into four phases: the pure HVs, the HV-dominated, the AVdominated, and the pure AVs. This paper is primarily focused on the latter three phases. It is the first-of-its-kind survey paper to comprehensively review literature in both transportation engineering and AI for mixed traffic modeling. Models used for each phase are summarized, encompassing game theory, deep (reinforcement) learning, and imitation learning. While reviewing the methodologies, we primarily focus on the following research questions: (1) What scalable driving policies are to control a large number of AVs in mixed traffic comprised of human drivers and uncontrollable AVs? (2) How do we estimate human driver behaviors? (3) How should the driving behavior of uncontrollable AVs be modeled in the environment? (4) How are the interactions between human drivers and autonomous vehicles characterized? Hopefully this paper will not only inspire our transportation community to rethink the conventional models that are developed in the data-shortage era, but also reach out to other disciplines, in particular robotics and machine learning, to join forces towards creating a safe and efficient mixed traffic ecosystem.

[1]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[2]  Etienne Perot,et al.  End-to-End Driving in a Realistic Racing Game with Deep Reinforcement Learning , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  M J Lighthill,et al.  On kinematic waves II. A theory of traffic flow on long crowded roads , 1955, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[4]  Meixin Zhu,et al.  Modeling car-following behavior on urban expressways in Shanghai: A naturalistic driving study , 2018, Transportation Research Part C: Emerging Technologies.

[5]  Meng Wang,et al.  Infrastructure assisted adaptive driving to stabilise heterogeneous vehicle strings , 2018, Transportation Research Part C: Emerging Technologies.

[6]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[7]  Meng Wang,et al.  Rolling horizon control framework for driver assistance systems. Part I: Mathematical formulation and non-cooperative systems , 2014 .

[8]  Jeremiah Singer,et al.  Travel Time on Arterials and Rural Highways: State-of-the-Practice Synthesis on Rural Data Collection Technology , 2013 .

[9]  Soyoung Ahn,et al.  Receding Horizon Stochastic Optimal Control Strategy for ACC and CACC under Uncertainty , 2017 .

[10]  G.K. Venayagamoorthy,et al.  Unmanned vehicle navigation using swarm intelligence , 2004, International Conference on Intelligent Sensing and Information Processing, 2004. Proceedings of.

[11]  Mitsuru Tanaka.  Development of Various Artificial Neural Network Car-Following Models with Converted Data Sets by A Self-Organization Neural Network , 2013 .

[12]  Zhe Xu,et al.  Efficient Large-Scale Fleet Management via Multi-Agent Deep Reinforcement Learning , 2018, KDD.

[13]  Matthew Johnson-Roberson,et al.  Driving in the Matrix: Can virtual worlds replace human-generated annotations for real world tasks? , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[15]  Petros A. Ioannou,et al.  A Comparision of Spacing and Headway Control Laws for Automatically Controlled Vehicles1 , 1994 .

[16]  Armin Mustafa,et al.  A*3D Dataset: Towards Autonomous Driving in Challenging Environments , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Reza Langari,et al.  Stackelberg Game Based Model of Highway Driving , 2012 .

[18]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[19]  Rüdiger Dillmann,et al.  A probabilistic model for estimating driver behaviors and vehicle trajectories in traffic environments , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[20]  Fawzi Nashashibi,et al.  A Cooperative Car-Following/Emergency Braking System With Prediction-Based Pedestrian Avoidance Capabilities , 2019, IEEE Transactions on Intelligent Transportation Systems.

[21]  Sergey Levine,et al.  Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor , 2018, ICML.

[22]  Hao Zhou,et al.  Longitudinal Motion Planning for Autonomous Vehicles and Its Impact on Congestion: A Survey , 2019, ArXiv.

[23]  Sergey Levine,et al.  High-Dimensional Continuous Control Using Generalized Advantage Estimation , 2015, ICLR.

[24]  Xuan Di,et al.  Liability Design for Autonomous Vehicles and Human-Driven Vehicles: A Hierarchical Game-Theoretic Approach , 2019, SSRN Electronic Journal.

[25]  Mykel J. Kochenderfer,et al.  Reinforcement Learning with Probabilistic Guarantees for Autonomous Driving , 2019, ArXiv.

[26]  Lili Du,et al.  Constrained optimization and distributed computation based car following control of a connected and autonomous vehicle platoon , 2016 .

[27]  Xuan Di,et al.  An LSTM-Based Autonomous Driving Model Using Waymo Open Dataset , 2020, ArXiv.

[28]  Yang Zhou,et al.  Robust local and string stability for a decentralized car following control strategy for connected automated vehicles , 2019, Transportation Research Part B: Methodological.

[29]  Reza Langari,et al.  Game theory based autonomous vehicles operation , 2014 .

[30]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[31]  Yu Wang,et al.  A trajectory smoothing method at signalized intersection based on individualized variable speed limits with location optimization , 2018 .

[32]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[33]  A. Dreves,et al.  A generalized Nash equilibrium approach for optimal control problems of autonomous cars , 2018 .

[34]  H. J. Van Zuylen,et al.  Bayesian Calibration of Car-Following Models , 2010, CTS 2009.

[35]  Guillaume J. Laurent,et al.  Independent reinforcement learners in cooperative Markov games: a survey regarding coordination problems , 2012, The Knowledge Engineering Review.

[36]  Henry X. Liu,et al.  A Game Theoretical Approach for Modelling Merging and Yielding Behavior at Freeway On-Ramp Sections , 2007 .

[37]  Stephen D. Boyles,et al.  Dynamic traffic assignment of cooperative adaptive cruise control , 2018 .

[38]  Mykel J. Kochenderfer,et al.  Belief state planning for autonomously navigating urban intersections , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[39]  Etienne Perot,et al.  Deep Reinforcement Learning framework for Autonomous Driving , 2017, Autonomous Vehicles and Machines.

[40]  Katherine Rose Driggs-Campbell,et al.  Simulating Emergent Properties of Human Driving Behavior Using Multi-Agent Reward Augmented Imitation Learning , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[41]  Stephen D. Boyles,et al.  Intersection Auctions and Reservation-Based Control in Dynamic Traffic Assignment , 2015 .

[42]  Indrajit Chatterjee,et al.  Evolutionary Game Theoretic Approach to Rear-End Events on Congested Freeway , 2013 .

[43]  Ruzena Bajcsy,et al.  Integrating Intuitive Driver Models in Autonomous Planning for Interactive Maneuvers , 2017, IEEE Transactions on Intelligent Transportation Systems.

[44]  Nan Li,et al.  Game-Theoretic Modeling of Traffic in Unsignalized Intersection Network for Autonomous Vehicle Control Verification and Validation , 2019, IEEE Transactions on Intelligent Transportation Systems.

[45]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[46]  Shimon Whiteson,et al.  Learning to Communicate with Deep Multi-Agent Reinforcement Learning , 2016, NIPS.

[47]  Stefano Ermon,et al.  Multi-Agent Generative Adversarial Imitation Learning , 2018, NeurIPS.

[48]  Hesham Rakha,et al.  Comparison of Greenshields, Pipes, and Van Aerde Car-Following and Traffic Stream Models , 2002 .

[49]  Hani S. Mahmassani,et al.  Modeling Lane-Changing Behavior in a Connected Environment: A Game Theory Approach , 2015 .

[50]  Gregory D. Hager,et al.  Combining neural networks and tree search for task and motion planning in challenging environments , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[51]  Wei Huang,et al.  Traffic parameters estimation for signalized intersections based on combined shockwave analysis and Bayesian Network , 2019, Transportation Research Part C: Emerging Technologies.

[52]  Dawn M. Tilbury,et al.  Pedestrian Trust in Automated Vehicles: Role of Traffic Signal and AV Driving Behavior , 2019, Front. Robot. AI.

[53]  Jeroen Ploeg,et al.  Consensus Control for Vehicular Platooning With Velocity Constraints , 2018, IEEE Transactions on Control Systems Technology.

[54]  Yi Wu,et al.  Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments , 2017, NIPS.

[55]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[56]  Sergey Levine,et al.  Goal-driven dynamics learning via Bayesian optimization , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[57]  Gábor Stépán,et al.  Traffic jams: dynamics and control , 2010, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[58]  Paris Perdikaris,et al.  Adversarial Uncertainty Quantification in Physics-Informed Neural Networks , 2018, J. Comput. Phys..

[59]  Maria Laura Delle Monache,et al.  Dissipation of stop-and-go waves via control of autonomous vehicles: Field experiments , 2017, ArXiv.

[60]  Serge P. Hoogendoorn,et al.  Generic Calibration Framework for Joint Estimation of Car-Following Models by Using Microscopic Data , 2010 .

[61]  Kuang Huang,et al.  Stabilizing Traffic via Autonomous Vehicles: A Continuum Mean Field Game Approach , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[62]  Xiaobo Qu,et al.  A piecewise trajectory optimization model for connected automated vehicles: Exact optimization algorithm and queue propagation analysis , 2018, Transportation Research Part B: Methodological.

[63]  Peng Hao,et al.  Eco-Approach and Departure (EAD) Application for Actuated Signals in Real-World Traffic , 2016 .

[64]  Ingo Wolf.  The Interaction Between Humans and Autonomous Agents , 2016 .

[65]  Alireza Talebpour,et al.  Influence of connected and autonomous vehicles on traffic flow stability and throughput , 2016 .

[66]  Stephen D. Boyles,et al.  A multiclass cell transmission model for shared human and autonomous vehicle roads , 2016 .

[67]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[68]  Yang Zheng,et al.  Dynamical Modeling and Distributed Control of Connected and Automated Vehicles: Challenges and Opportunities , 2017, IEEE Intelligent Transportation Systems Magazine.

[69]  Anca D. Dragan,et al.  Hierarchical Game-Theoretic Planning for Autonomous Vehicles , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[70]  Alexander Skabardonis,et al.  Freeway Traffic Shockwave Analysis: Exploring NGSIM Trajectory Data , 2007 .

[71]  Meng Wang,et al.  Rolling horizon control framework for driver assistance systems. Part II: Cooperative sensing and cooperative control , 2014 .

[72]  Alexandre M. Bayen,et al.  Dissipating stop-and-go waves in closed and open networks via deep reinforcement learning , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[73]  Behdad Chalaki,et al.  Simulation to scaled city: zero-shot policy transfer for traffic control via autonomous vehicles , 2018, ICCPS.

[74]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[75]  Robert E. Schapire,et al.  A Game-Theoretic Approach to Apprenticeship Learning , 2007, NIPS.

[76]  Fangyu Wu,et al.  Connections between classical car following models and artificial neural networks , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[77]  Le Yi Wang,et al.  Platoon Control of Connected Vehicles from a Networked Control Perspective: Literature Review, Component Modeling, and Controller Synthesis , 2018 .

[78]  Markos Papageorgiou,et al.  Simulation of the penetration rate effects of ACC and CACC on macroscopic traffic dynamics , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[79]  Hesham M. Eraqi,et al.  End-to-End Deep Learning for Steering Autonomous Vehicles Considering Temporal Dependencies , 2017, ArXiv.

[80]  Christos Katrakazas,et al.  Real-time motion planning methods for autonomous on-road driving: State-of-the-art and future research directions , 2015 .

[81]  Peter Stone,et al.  Sharing the Road: Autonomous Vehicles Meet Human Drivers , 2007, IJCAI.

[82]  P. Lions,et al.  Mean field games , 2007 .

[83]  João Pedro Hespanha,et al.  Mistuning-Based Control Design to Improve Closed-Loop Stability Margin of Vehicular Platoons , 2008, IEEE Transactions on Automatic Control.

[84]  Heechul Yun,et al.  DeepPicar: A Low-Cost Deep Neural Network-Based Autonomous Car , 2017, 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA).

[85]  Fawzi Nashashibi,et al.  End-to-End Race Driving with Deep Reinforcement Learning , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[86]  Indrajit Chatterjee,et al.  Understanding Driver Contributions to Rear-End Crashes on Congested Freeways and their Implications for Future Safety Measures , 2016 .

[87]  Xiaobo Qu,et al.  A recurrent neural network based microscopic car following model to predict traffic oscillation , 2017 .

[88]  Zhensong Wei,et al.  Vision-Based Lane-Changing Behavior Detection Using Deep Residual Neural Network , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[89]  Kuang Huang,et al.  A Game-Theoretic Framework for Autonomous Vehicles Velocity Control: Bridging Microscopic Differential Games and Macroscopic Mean Field Games , 2019, Discrete & Continuous Dynamical Systems - B.

[90]  Victor Talpaert,et al.  Deep Reinforcement Learning for Autonomous Driving: A Survey , 2020, IEEE Transactions on Intelligent Transportation Systems.

[91]  Jonathan M. Hankey,et al.  Description of the SHRP 2 Naturalistic Database and the Crash, Near-Crash, and Baseline Data Sets , 2016 .

[92]  Malte Risto,et al.  The social behavior of autonomous vehicles , 2016, UbiComp Adjunct.

[93]  Carlos F. Daganzo,et al.  TRANSPORTATION AND TRAFFIC THEORY , 1993 .

[94]  Tamer Basar,et al.  Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms , 2019, Handbook of Reinforcement Learning and Control.

[95]  Gábor Orosz,et al.  Dynamics of connected vehicle systems with delayed acceleration feedback , 2014 .

[96]  Gábor Orosz,et al.  Digital Effects and Delays in Connected Vehicles: Linear Stability and Simulations , 2013 .

[97]  Masayoshi Tomizuka,et al.  Model-free Deep Reinforcement Learning for Urban Autonomous Driving , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[98]  Hamidou Tembine,et al.  Mean-Field-Type Games in Engineering , 2016, ArXiv.

[99]  Serge P. Hoogendoorn,et al.  Continuum modeling of cooperative traffic flow dynamics , 2009 .

[100]  Ulrich Kressel,et al.  Probabilistic trajectory prediction with Gaussian mixture models , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[101]  Stefano Ermon,et al.  Model-Free Imitation Learning with Policy Optimization , 2016, ICML.

[102]  Christian Laugier,et al.  High-speed highway scene prediction based on driver models learned from demonstrations , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[103]  Hesham Rakha,et al.  Comparison and calibration of FRESIM and INTEGRATION steady-state car-following behavior , 2003 .

[104]  Xuesong Zhou,et al.  Dynamic programming-based multi-vehicle longitudinal trajectory optimization with simplified car following models , 2017 .

[105]  Pål Andreas Pedersen,et al.  A Game Theoretical Approach to Road Safety , 2001 .

[106]  Kaan Ozbay,et al.  New Calibration Methodology for Microscopic Traffic Simulation Using Enhanced Simultaneous Perturbation Stochastic Approximation Approach , 2009 .

[107]  Hesham Rakha,et al.  Calibration of Steady-State Car-Following Models Using Macroscopic Loop Detector Data , 2010 .

[108]  Nan Li,et al.  Adaptive Game-Theoretic Decision Making for Autonomous Vehicle Control at Roundabouts , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[109]  Stefano Ermon,et al.  Generative Adversarial Imitation Learning , 2016, NIPS.

[110]  Xiang Wang,et al.  Background Filtering and Vehicle Detection with Roadside Lidar Based on Point Association , 2018, 2018 37th Chinese Control Conference (CCC).

[111]  Jaime Fernandez Fisac,et al.  Game-Theoretic Safety Assurance for Human-Centered Robotic Systems , 2019 .

[112]  X. Jessie Yang,et al.  Analysis and Prediction of Pedestrian Crosswalk Behavior during Automated Vehicle Interactions , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[113]  Yang Gao,et al.  End-to-End Learning of Driving Models from Large-Scale Video Datasets , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[114]  Daniel J. Fagnant,et al.  An Assessment of Autonomous Vehicles: Traffic Impacts and Infrastructure Needs—Final Report , 2017 .

[115]  Alireza Talebpour,et al.  A Platooning Strategy for Automated Vehicles in the Presence of Speed Limit Fluctuations , 2018, Transportation Research Record: Journal of the Transportation Research Board.

[116]  Hesham A. Rakha,et al.  Freeway Speed Harmonization , 2016, IEEE Transactions on Intelligent Vehicles.

[117]  Jürgen Schmidhuber,et al.  Recurrent policy gradients , 2010, Log. J. IGPL.

[118]  Marie-Therese Wolfram,et al.  On a mean field game approach modeling congestion and aversion in pedestrian crowds , 2011 .

[119]  Chaozhe R. He,et al.  Experimental validation of connected automated vehicle design among human-driven vehicles , 2018, Transportation Research Part C: Emerging Technologies.

[120]  Masayoshi Tomizuka,et al.  Safe exploration: Addressing various uncertainty levels in human robot interactions , 2015, 2015 American Control Conference (ACC).

[121]  Yafeng Yin,et al.  Optimal deployment of autonomous vehicle lanes with endogenous market penetration , 2016 .

[122]  Gábor Orosz,et al.  Scalable stability analysis on large connected vehicle systems subject to stochastic communication delays , 2017 .

[123]  Wojciech M. Czarnecki,et al.  Grandmaster level in StarCraft II using multi-agent reinforcement learning , 2019, Nature.

[124]  Mohamed M. Ahmed,et al.  Evaluation of weather-related freeway car-following behavior using the SHRP2 naturalistic driving study database , 2018, Transportation Research Part F: Traffic Psychology and Behaviour.

[125]  Jin Zhang,et al.  On the fundamental diagram for freeway traffic: A novel calibration approach for single-regime models , 2015 .

[126]  Yuchuan Du,et al.  Optimal design of autonomous vehicle zones in transportation networks , 2017 .

[127]  Serge P. Hoogendoorn,et al.  Heterogeneity In Car-Following Behavior: Theory And Empirics , 2011 .

[128]  Wolfgang Rosenstiel,et al.  Object-oriented Bayesian networks for detection of lane change maneuvers , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[129]  Sergey Levine,et al.  Learning deep control policies for autonomous aerial vehicles with MPC-guided policy search , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[130]  V. Manera,et al.  Grasping intentions: from thought experiments to empirical evidence , 2012, Front. Hum. Neurosci..

[131]  Bart van Arem,et al.  Effects of Cooperative Adaptive Cruise Control on traffic flow stability , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[132]  Nan Li,et al.  Game Theoretic Modeling of Driver and Vehicle Interactions for Verification and Validation of Autonomous Vehicle Control Systems , 2016, IEEE Transactions on Control Systems Technology.

[133]  Hussein Dia,et al.  Neural Agent Car-Following Models , 2007, IEEE Transactions on Intelligent Transportation Systems.

[134]  Ali Ghaffari,et al.  A Modified Car-Following Model Based on a Neural Network Model of the Human Driver Effects , 2012, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[135]  Reza Langari,et al.  A Game-Theoretic Model of Human Driving and Application to Discretionary Lane-Changes , 2020, ArXiv.

[136]  Baher Abdulhai,et al.  Genetic Algorithm-Based Optimization Approach and Generic Tool for Calibrating Traffic Microscopic Simulation Parameters , 2002 .

[137]  P. I. Richards.  Shock Waves on the Highway , 1956 .

[138]  Ruzena Bajcsy,et al.  Communicating intent on the road through human-inspired control schemes , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[139]  Cathy Wu,et al.  Learning and Optimization for Mixed Autonomy Systems - A Mobility Context , 2018 .

[140]  Vladlen Koltun,et al.  Playing for Benchmarks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[141]  Alireza Talebpour,et al.  Towards a collaborative connected, automated driving environment: A game theory based decision framework for unprotected left turn maneuvers , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[142]  George E. Karniadakis,et al.  Hidden physics models: Machine learning of nonlinear partial differential equations , 2017, J. Comput. Phys..

[143]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[144]  Reza Langari,et al.  A human-like game theory-based controller for automatic lane changing , 2018 .

[145]  D. P. Greveling.  Modelling human driving behaviour using Generative Adversarial Networks , 2018 .

[146]  Adam Millard-Ball,et al.  Pedestrians, Autonomous Vehicles, and Cities , 2016 .

[147]  Eugene Vinitsky,et al.  Flow: A Modular Learning Framework for Autonomy in Traffic. , 2017 .

[148]  Melissa Cefkin,et al.  Developing Socially Acceptable Autonomous Vehicles , 2016 .

[149]  Maarten Steinbuch,et al.  String-Stable CACC Design and Experimental Validation: A Frequency-Domain Approach , 2010, IEEE Transactions on Vehicular Technology.

[150]  Sorin Grigorescu,et al.  A Survey of Deep Learning Techniques for Autonomous Driving , 2020, J. Field Robotics.

[151]  Chang Liu,et al.  Learning a deep neural net policy for end-to-end control of autonomous vehicles , 2017, 2017 American Control Conference (ACC).

[152]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[153]  Mashrur Chowdhury,et al.  Improving the Efficacy of Car-Following Models With a New Stochastic Parameter Estimation and Calibration Method , 2015, IEEE Transactions on Intelligent Transportation Systems.

[154]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[155]  Anca D. Dragan,et al.  Planning for Autonomous Cars that Leverage Effects on Human Actions , 2016, Robotics: Science and Systems.

[156]  Yann LeCun,et al.  Off-Road Obstacle Avoidance through End-to-End Learning , 2005, NIPS.

[157]  G. F. Newell.  Nonlinear Effects in the Dynamics of Car Following , 1961 .

[158]  Michel Rascle,et al.  Resurrection of "Second Order" Models of Traffic Flow , 2000, SIAM J. Appl. Math..

[159]  Dong Ngoduy,et al.  Instability of cooperative adaptive cruise control traffic flow: A macroscopic approach , 2013, Commun. Nonlinear Sci. Numer. Simul..

[160]  Serge P. Hoogendoorn,et al.  Stabilizing mixed vehicular platoons with connected automated vehicles: An H-infinity approach , 2020, Transportation Research Part B: Methodological.

[161]  Yong Tang,et al.  Vehicle detection and recognition for intelligent traffic surveillance system , 2017, Multimedia Tools and Applications.

[162]  Dong Ngoduy,et al.  Analytical studies on the instabilities of heterogeneous intelligent traffic flow , 2013, Commun. Nonlinear Sci. Numer. Simul..

[163]  Nathan van de Wouw,et al.  Design and experimental evaluation of cooperative adaptive cruise control , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[164]  Stephen D. Boyles,et al.  Effects of Autonomous Vehicle Behavior on Arterial and Freeway Networks , 2016 .

[165]  Carl-Johan Hoel,et al.  Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving , 2019, IEEE Transactions on Intelligent Vehicles.

[166]  Stephen Graham Ritchie,et al.  TRANSPORTATION RESEARCH. PART C, EMERGING TECHNOLOGIES , 1993 .

[167]  George Em Karniadakis,et al.  Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations , 2020, Science.

[168]  Mykel J. Kochenderfer,et al.  Imitating driver behavior with generative adversarial networks , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[169]  Mark D. Miller,et al.  Modeling Effects of Driver Control Assistance Systems on Traffic , 2001 .

[170]  Qingyu Zhang,et al.  Receding Horizon Markov Game Autonomous Driving Strategy , 2019, 2019 American Control Conference (ACC).

[171]  David Silver,et al.  Memory-based control with recurrent neural networks , 2015, ArXiv.

[172]  Swaroop Darbha,et al.  Intelligent Cruise Control Systems And Traffic Flow Stability , 1998 .

[173]  Le Yi Wang,et al.  Stability Margin Improvement of Vehicular Platoon Considering Undirected Topology and Asymmetric Control , 2016, IEEE Transactions on Control Systems Technology.

[174]  R. Cabeza,et al.  Frontiers in Human Neuroscience , 2009 .

[175]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[176]  Ruzena Bajcsy,et al.  Robust, Informative Human-in-the-Loop Predictions via Empirical Reachable Sets , 2017, IEEE Transactions on Intelligent Vehicles.

[177]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[178]  Andreas A. Malikopoulos,et al.  Optimal Control for Speed Harmonization of Automated Vehicles , 2016, IEEE Transactions on Intelligent Transportation Systems.

[179]  Harold J Payne,et al.  MODELS OF FREEWAY TRAFFIC AND CONTROL. , 1971 .

[180]  Soyoung Ahn,et al.  Distributed model predictive control approach for cooperative car-following with guaranteed local and string stability , 2019, Transportation Research Part B: Methodological.

[181]  J. Hedrick,et al.  String stability of interconnected systems , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[182]  P. A. Pedersen.  Moral Hazard in Traffic Games , 2003 .

[183]  Ruigang Yang,et al.  ApolloCar3D: A Large 3D Car Instance Understanding Benchmark for Autonomous Driving , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[184]  Danya Yao,et al.  A Survey of Traffic Control With Vehicular Communications , 2014, IEEE Transactions on Intelligent Transportation Systems.

[185]  Lawrence D. Jackel,et al.  Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car , 2017, ArXiv.

[186]  Atsushi Yamashita,et al.  Lane-Change Detection Based on Vehicle-Trajectory Prediction , 2017, IEEE Robotics and Automation Letters.

[187]  Daniela Rus,et al.  Learning Robust Control Policies for End-to-End Autonomous Driving From Data-Driven Simulation , 2020, IEEE Robotics and Automation Letters.

[188]  Wenshuo Wang,et al.  Feature analysis and selection for training an end-to-end autonomous vehicle controller using deep learning approach , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[189]  Jonathan P. How,et al.  Driver Behavior Classification at Intersections and Validation on Large Naturalistic Data Set , 2012, IEEE Transactions on Intelligent Transportation Systems.

[190]  P. G. Gipps,et al.  A behavioural car-following model for computer simulation , 1981 .

[191]  William H. K. Lam,et al.  Transportation Research, Part A: Policy and Practice , 1997 .

[192]  Peter E. Caines,et al.  Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle , 2006, Commun. Inf. Syst..

[193]  Vladlen Koltun,et al.  Playing for Data: Ground Truth from Computer Games , 2016, ECCV.

[194]  Robert Bogue.  Swarm intelligence and robotics , 2008, Ind. Robot.

[195]  Saeid Nahavandi,et al.  Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications , 2018, IEEE Transactions on Cybernetics.

[196]  Gaetano Fusco,et al.  Artificial Neural Network Models for Car Following: Experimental Analysis and Calibration Issues , 2014, J. Intell. Transp. Syst..

[197]  John Fox,et al.  The Knowledge Engineering Review , 1984, The Knowledge Engineering Review.

[198]  Byron Boots,et al.  Agile Autonomous Driving using End-to-End Deep Imitation Learning , 2017, Robotics: Science and Systems.

[199]  YangQuan Chen,et al.  Formation control: a review and a new consideration , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[200]  Gary A. Davis.  Bayesian Estimation of Drivers’ Gap Selections and Reaction Times in Left-Turning Crashes from Event Data Recorder Pre-Crash Data , 2017 .

[201]  Yi-Ting Chen,et al.  The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[202]  Srinivas Peeta,et al.  Consensus-Based Cooperative Control for Multi-Platoon Under the Connected Vehicles Environment , 2019, IEEE Transactions on Intelligent Transportation Systems.

[203]  A. Lachapelle,et al.  COMPUTATION OF MEAN FIELD EQUILIBRIA IN ECONOMICS , 2010 .

[204]  Kikuo Fujimura,et al.  Tactical Decision Making for Lane Changing with Deep Reinforcement Learning , 2017 .

[205]  Dorsa Sadigh,et al.  Batch Active Preference-Based Learning of Reward Functions , 2018, CoRL.

[206]  Masayoshi Tomizuka,et al.  Enabling safe freeway driving for automated vehicles , 2016, 2016 American Control Conference (ACC).

[207]  Alexandre M. Bayen,et al.  Evaluation of traffic data obtained via GPS-enabled mobile phones: The Mobile Century field experiment , 2009 .

[208]  Sanjit A. Seshia,et al.  Verifying Robustness of Human-Aware Autonomous Cars , 2019, IFAC-PapersOnLine.

[209]  Bart van Arem,et al.  The Impact of Cooperative Adaptive Cruise Control on Traffic-Flow Characteristics , 2006, IEEE Transactions on Intelligent Transportation Systems.

[210]  Rahul Savani,et al.  Lenient Multi-Agent Deep Reinforcement Learning , 2017, AAMAS.

[211]  C. Urmson,et al.  Classification and tracking of dynamic objects with multiple sensors for autonomous driving in urban environments , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[212]  Masayoshi Tomizuka,et al.  Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safety , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[213]  Soyoung Ahn,et al.  Traffic dynamics under speed disturbance in mixed traffic with automated and non-automated vehicles , 2019 .

[214]  Shane B. McLaughlin,et al.  Naturalistic Driving Study: Linking the Study Data to the Roadway Information Database , 2015 .

[215]  Steven E Shladover,et al.  Modeling cooperative and autonomous adaptive cruise control dynamic responses using experimental data , 2014 .

[216]  Luc Van Gool,et al.  End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners , 2018, ECCV.

[217]  Alexandre M. Bayen,et al.  Calibration Framework based on Bluetooth Sensors for Traffic State Estimation Using a Velocity based Cell Transmission Model , 2014 .

[218]  Jonathan P. How,et al.  Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability , 2017, ICML.

[219]  Mykel J. Kochenderfer,et al.  Multi-Agent Imitation Learning for Driving Simulation , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[220]  Cynthia Breazeal,et al.  Machine behaviour , 2019, Nature.

[221]  Mohan M. Trivedi,et al.  Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior Analysis , 2013, IEEE Transactions on Intelligent Transportation Systems.

[222]  In So Kweon,et al.  KAIST Multi-Spectral Day/Night Data Set for Autonomous and Assisted Driving , 2018, IEEE Transactions on Intelligent Transportation Systems.

[223]  Masayoshi Tomizuka,et al.  Improving Efficiency of Autonomous Vehicles by V2V Communication , 2018, 2018 Annual American Control Conference (ACC).

[224]  Jianxiong Xiao,et al.  DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[225]  Gábor Orosz,et al.  Connected cruise control among human-driven vehicles: Experiment-based parameter estimation and optimal control design , 2018, Transportation Research Part C: Emerging Technologies.

[226]  Makoto Kasai,et al.  Application of hierarchical Bayesian estimation to calibrating a car-following model with time-varying parameters , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[227]  Maziar Raissi,et al.  Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations , 2018, J. Mach. Learn. Res..

[228]  Rajesh Rajamani,et al.  An Experimental Comparative Study of Autonomous and Co-operative Vehicle-follower Control Systems , 2001 .

[229]  Dorian Kodelja,et al.  Multiagent cooperation and competition with deep reinforcement learning , 2015, PloS one.

[230]  Noam Brown,et al.  Superhuman AI for multiplayer poker , 2019, Science.

[231]  Dorsa Sadigh,et al.  Maximizing Road Capacity Using Cars that Influence People , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[232]  Qiang Xu,et al.  nuScenes: A Multimodal Dataset for Autonomous Driving , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[233]  Noam Brown,et al.  Superhuman AI for heads-up no-limit poker: Libratus beats top professionals , 2018, Science.

[234]  J. Andrew Bagnell,et al.  Efficient Reductions for Imitation Learning , 2010, AISTATS.

[235]  Srinivas Peeta,et al.  Nonlinear Consensus-Based Connected Vehicle Platoon Control Incorporating Car-Following Interactions and Heterogeneous Time Delays , 2019, IEEE Transactions on Intelligent Transportation Systems.

[236]  Lili Du,et al.  Cooperative platoon control for a mixed traffic flow including human drive vehicles and connected and autonomous vehicles , 2018, Transportation Research Part B: Methodological.

[237]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[238]  Berthold Färber,et al.  Communication and Communication Problems Between Autonomous Vehicles and Human Drivers , 2016 .

[239]  Reza Langari,et al.  A Stackelberg Game Theoretic Driver Model for Merging , 2013 .

[240]  Meng Wang,et al.  Game theoretic approach for predictive lane-changing and car-following control , 2015 .

[241]  Helbing,et al.  Congested traffic states in empirical observations and microscopic simulations , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[242]  Mayank Bansal,et al.  ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst , 2018, Robotics: Science and Systems.

[243]  Sergey Levine,et al.  Continuous Inverse Optimal Control with Locally Optimal Examples , 2012, ICML.

[244]  Ravishankar K. Iyer,et al.  Experimental evaluation , 1995 .

[245]  Mykel J. Kochenderfer,et al.  The value of inferring the internal state of traffic participants for autonomous freeway driving , 2017, 2017 American Control Conference (ACC).

[246]  Andreas A. Malikopoulos,et al.  A Survey on the Coordination of Connected and Automated Vehicles at Intersections and Merging at Highway On-Ramps , 2017, IEEE Transactions on Intelligent Transportation Systems.

[247]  Roberto Horowitz,et al.  Fundamental Diagram Calibration: A Stochastic Approach to Linear Fitting , 2014 .

[248]  Dirk Helbing,et al.  Enhanced intelligent driver model to access the impact of driving strategies on traffic capacity , 2009, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[249]  Alexandre M. Bayen,et al.  Emergent Behaviors in Mixed-Autonomy Traffic , 2017, CoRL.

[250]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[251]  Xiao-Yun Lu,et al.  COOPERATIVE ADAPTIVE CRUISE CONTROL (CACC) DEFINITIONS AND OPERATING CONCEPTS , 2015 .

[252]  Alexey Dosovitskiy,et al.  End-to-End Driving Via Conditional Imitation Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[253]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[254]  Michael S. Triantafyllou,et al.  Deep learning of vortex-induced vibrations , 2018, Journal of Fluid Mechanics.

[255]  Melissa Cefkin,et al.  Evaluation of an Autonomous Vehicle External Communication System Concept: A Survey Study , 2017 .

[256]  Mikhail Gordon,et al.  Lane Change and Merge Maneuvers for Connected and Automated Vehicles: A Survey , 2016, IEEE Transactions on Intelligent Vehicles.

[257]  Christos Dimitrakakis,et al.  TORCS, The Open Racing Car Simulator , 2005 .

[258]  Jie Sun,et al.  A car-following model considering asymmetric driving behavior based on long short-term memory neural networks , 2018, Transportation Research Part C: Emerging Technologies.

[259]  Meng Wang,et al.  Cooperative Car-Following Control: Distributed Algorithm and Impact on Moving Jam Features , 2016, IEEE Transactions on Intelligent Transportation Systems.

[260]  Alexandre M. Bayen,et al.  Benchmarks for reinforcement learning in mixed-autonomy traffic , 2018, CoRL.

[261]  Alexandre M. Bayen,et al.  Framework for control and deep reinforcement learning in traffic , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[262]  Alexandre M. Bayen,et al.  Arterial travel time forecast with streaming data: A hybrid approach of flow modeling and machine learning , 2012 .

[263]  Xuan Di,et al.  Scalable traffic stability analysis in mixed-autonomy using continuum models , 2020 .

[264]  Motoyuki Akamatsu,et al.  Prediction of Human Driving Behavior Using Dynamic Bayesian Networks , 2006, IEICE Trans. Inf. Syst..

[265]  Mathias Perrollaz,et al.  Learning-based approach for online lane change intention prediction , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[266]  Wang Yi,et al.  Stability analysis and the fundamental diagram for mixed connected automated and human-driven vehicles , 2019, Physica A: Statistical Mechanics and its Applications.

[267]  Qingyu Zhang,et al.  A Game Theoretic Model Predictive Controller With Aggressiveness Estimation for Mandatory Lane Change , 2020, IEEE Transactions on Intelligent Vehicles.

[268]  Simon Lucey,et al.  Argoverse: 3D Tracking and Forecasting With Rich Maps , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[269]  Meixin Zhu,et al.  Human-Like Autonomous Car-Following Planning by Deep Reinforcement Learning , 2018 .

[270]  Fei-Yue Wang,et al.  Capturing Car-Following Behaviors by Deep Learning , 2018, IEEE Transactions on Intelligent Transportation Systems.

[271]  Guoyuan Wu,et al.  GlidePath: Eco-Friendly Automated Approach and Departure at Signalized Intersections , 2017, IEEE Transactions on Intelligent Vehicles.

[272]  Keqiang Li,et al.  Lane changing intention recognition based on speech recognition models , 2016 .

[273]  Dirk Helbing,et al.  General Lane-Changing Model MOBIL for Car-Following Models , 2007 .

[274]  Swaroop Darbha,et al.  Direct adaptive longitudinal control of vehicle platoons , 2001, IEEE Trans. Veh. Technol..

[275]  Luc Van Gool,et al.  Learning Driving Models with a Surround-View Camera System and a Route Planner , 2018, ArXiv.

[276]  Yu Wang,et al.  Field experiments on longitudinal characteristics of human driver behavior following an autonomous vehicle , 2020 .

[277]  Elsevier Sdol.  Transportation Research Part F: Traffic Psychology and Behaviour , 2009 .

[278]  Ulrich Weidmann,et al.  Assessing the feasibility of transport Megaprojects; ; Transportation research record : journal of the Transportation Research Board; , 2007 .

[279]  Benjamin Seibold,et al.  Stabilizing traffic flow via a single autonomous vehicle: Possibilities and limitations , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[280]  Alexandre M. Bayen,et al.  Flow: Architecture and Benchmarking for Reinforcement Learning in Traffic Control , 2017, ArXiv.

[281]  Xuan Di,et al.  Long-Term Prediction of Lane Change Maneuver Through a Multilayer Perceptron , 2020, 2020 IEEE Intelligent Vehicles Symposium (IV).

[282]  Jun Wang,et al.  Efficient Ridesharing Order Dispatching with Mean Field Multi-Agent Reinforcement Learning , 2019, WWW.

[283]  Alexandre M. Bayen,et al.  Stabilizing Traffic with Autonomous Vehicles , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[284]  Vicente Milanés Montero,et al.  Cooperative Adaptive Cruise Control in Real Traffic Situations , 2014, IEEE Transactions on Intelligent Transportation Systems.

[285]  Petros A. Ioannou,et al.  Autonomous intelligent cruise control , 1993 .