Annual Review of Control , Robotics , and Autonomous Systems Planning and Decision-Making for Autonomous Vehicles

In this review, we provide an overview of emerging trends and challenges in the field of intelligent and autonomous, or self-driving, vehicles. Recent advances in the field of perception, planning, and decision-making for autonomous vehicles have led to great improvements in functional capabilities, with several prototypes already driving on our roads and streets. Yet challenges remain regarding guaranteed performance and safety under all driving circumstances. For instance, planning methods that provide safe and systemcompliant performance in complex, cluttered environments while modeling the uncertain interaction with other traffic participants are required. Furthermore, new paradigms, such as interactive planning and end-to-end learning, open up questions regarding safety and reliability that need to be addressed. In this survey, we emphasize recent approaches for integrated perception and planning and for behavior-aware planning, many of which rely on machine learning. This raises the question of verification and safety, which we also touch upon. Finally, we discuss the state of the art and remaining challenges for managing fleets of autonomous vehicles. 187 A nn u. R ev . C on tr ol R ob ot . A ut on . S ys t. 20 18 .1 :1 87 -2 10 . D ow nl oa de d fr om w w w .a nn ua lr ev ie w s. or g A cc es s pr ov id ed b y D el ft U ni ve rs ity o f T ec hn ol og y on 0 5/ 30 /1 8. F or p er so na l u se o nl y. AS01CH08_Rus ARI 11 April 2018 20:0

[1]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[2]  Giuseppe Oriolo,et al.  Feedback control of a nonholonomic car-like robot , 1998 .

[3]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  O. Nelles.  Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuzzy Models , 2000 .

[5]  S. LaValle,et al.  Randomized Kinodynamic Planning , 2001 .

[6]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[7]  Rajesh Rajamani,et al.  Vehicle dynamics and control , 2005 .

[8]  Yann LeCun,et al.  Off-Road Obstacle Avoidance through End-to-End Learning , 2005, NIPS.

[9]  J. Andrew Bagnell,et al.  Maximum margin planning , 2006, ICML.

[10]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[11]  C.J. Tomlin,et al.  Autonomous Automobile Trajectory Tracking for Off-Road Driving: Controller Design, Experimental Validation and Racing , 2007, 2007 American Control Conference.

[12]  Francesco Borrelli,et al.  Predictive Active Steering Control for Autonomous Vehicle Systems , 2007, IEEE Transactions on Control Systems Technology.

[13]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[14]  Sebastian Thrun,et al.  Apprenticeship learning for motion planning with application to parking lot navigation , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Luke Fletcher,et al.  A perception-driven autonomous urban vehicle , 2008 .

[16]  Maxim Likhachev,et al.  Motion planning in urban environments , 2008 .

[17]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[18]  Sanjiv Singh,et al.  The DARPA Urban Challenge: Autonomous Vehicles in City Traffic, George Air Force Base, Victorville, California, USA , 2009, The DARPA Urban Challenge.

[19]  William Whittaker,et al.  Autonomous driving in urban environments: Boss and the Urban Challenge , 2008, J. Field Robotics.

[20]  Jonathan P. How,et al.  Real-Time Motion Planning With Applications to Autonomous Urban Driving , 2009, IEEE Transactions on Control Systems Technology.

[21]  Ross A. Knepper,et al.  Differentially constrained mobile robot motion planning in state lattices , 2009 .

[22]  Sebastian Thrun,et al.  Junior: The Stanford entry in the Urban Challenge , 2008, J. Field Robotics.

[23]  Niels A. H. Agatz,et al.  The Value of Optimization in Dynamic Ride-Sharing: A Simulation Study in Metro Atlanta , 2010 .

[24]  Tichakorn Wongpiromsarn,et al.  Formal Methods for Design and Verification of Embedded Control Systems: Application to an Autonomous Vehicle , 2010 .

[25]  David Silver,et al.  Learning from Demonstration for Autonomous Navigation in Complex Unstructured Terrain , 2010, Int. J. Robotics Res..

[26]  Ronen Lerner,et al.  Recent progress in road and lane detection: a survey , 2012, Machine Vision and Applications.

[27]  Emilio Frazzoli,et al.  Sampling-based algorithms for optimal motion planning , 2011, Int. J. Robotics Res..

[28]  André Platzer,et al.  Adaptive Cruise Control: Hybrid, Distributed, and Now Formally Verified , 2011, FM.

[29]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[30]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[31]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[32]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[33]  Han-Pang Huang,et al.  Robot Motion Planning in Dynamic Uncertain Environments , 2011, Adv. Robotics.

[34]  Mark Mulder,et al.  Haptic shared control: smoothly shifting control authority? , 2012, Cognition, Technology & Work.

[35]  Emilio Frazzoli,et al.  Robotic load balancing for mobility-on-demand systems , 2012, Int. J. Robotics Res..

[36]  Julius Ziegler,et al.  Optimal trajectories for time-critical street scenarios using discretized terminal manifolds , 2012, Int. J. Robotics Res..

[37]  David Silver,et al.  Learning Autonomous Driving Styles and Maneuvers from Expert Demonstration , 2012, ISER.

[38]  Sergey Levine,et al.  Continuous Inverse Optimal Control with Locally Optimal Examples , 2012, ICML.

[39]  Emilio Frazzoli,et al.  Least-violating control strategy synthesis with safety rules , 2013, HSCC '13.

[40]  Sterling J. Anderson,et al.  The intelligent copilot: A constraint-based approach to shared-adaptive control of ground vehicles , 2013, IEEE Intelligent Transportation Systems Magazine.

[41]  Michel Gendreau,et al.  A review of dynamic vehicle routing problems , 2013, Eur. J. Oper. Res..

[42]  John M. Dolan,et al.  Autonomous vehicle social behavior for highway entrance ramp management , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[43]  Alonzo Kelly,et al.  Vehicle model identification by integrated prediction error minimization , 2013, Int. J. Robotics Res..

[44]  Roland Siegwart,et al.  Toward automated driving in cities using close-to-market sensors: An overview of the V-Charge Project , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[45]  Rüdiger Dillmann,et al.  Probabilistic decision-making under uncertainty for autonomous driving using continuous POMDPs , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[46]  Paolo Santi,et al.  Supporting Information for Quantifying the Benefits of Vehicle Pooling with Shareability Networks Data Set and Pre-processing , 2022 .

[47]  Wilko Schwarting,et al.  Recursive conflict resolution for cooperative motion planning in dynamic highway traffic , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[48]  Patrick Pascheka,et al.  Cooperative decentralized decision making for conflict resolution among autonomous agents , 2014, 2014 IEEE International Symposium on Innovations in Intelligent Systems and Applications (INISTA) Proceedings.

[49]  Ruzena Bajcsy,et al.  Semiautonomous Vehicular Control Using Driver Modeling , 2014, IEEE Transactions on Intelligent Transportation Systems.

[50]  Paul A. Beardsley,et al.  Shared control of autonomous vehicles based on velocity space optimization , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[51]  Matthias Althoff,et al.  Online Verification of Automated Road Vehicles Using Reachability Analysis , 2014, IEEE Transactions on Robotics.

[52]  Markus Maurer,et al.  Towards Tactical Lane Change Behavior Planning for Automated Vehicles , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[53]  Marcelo H. Ang,et al.  Situation-aware decision making for autonomous driving on urban road using online POMDP , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[54]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Wolfram Burgard,et al.  Learning driving styles for autonomous vehicles from demonstration , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[56]  Christos Katrakazas,et al.  Real-time motion planning methods for autonomous on-road driving: State-of-the-art and future research directions , 2015 .

[57]  Markus Wulfmeier,et al.  Maximum Entropy Deep Inverse Reinforcement Learning , 2015, 1507.04888.

[58]  Ji Zhang,et al.  Visual-lidar odometry and mapping: low-drift, robust, and fast , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[59]  Sanjit A. Seshia,et al.  Compositional controller synthesis for vehicular traffic networks , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[60]  Markus Maurer,et al.  Structuring Cooperative Behavior Planning Implementations for Automated Driving , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[61]  Wolfram Burgard,et al.  Inverse reinforcement learning of behavioral models for online-adapting navigation strategies , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[62]  J. Christian Gerdes,et al.  Design of a feedback-feedforward steering controller for accurate path tracking and stability at the limits of handling , 2015 .

[63]  Jianxiong Xiao,et al.  DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[64]  Jörg Stückler,et al.  Large-scale direct SLAM with stereo cameras , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[65]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[66]  Andreas Krause,et al.  Robot navigation in dense human crowds: Statistical models and experimental studies of human–robot cooperation , 2015, Int. J. Robotics Res..

[67]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[68]  Paulo Tabuada,et al.  Correct-by-Construction Adaptive Cruise Control: Two Approaches , 2016, IEEE Transactions on Control Systems Technology.

[69]  Bart van Arem,et al.  Solving the User Optimum Privately Owned Automated Vehicles Assignment Problem (UO-POAVAP): A model to explore the impacts of self-driving vehicles on urban mobility , 2016 .

[70]  Hermann Winner,et al.  Autonomous Driving: Technical, Legal and Social Aspects , 2016 .

[71]  Anca D. Dragan,et al.  Information gathering actions over human internal state , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[72]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[73]  Stefano Ermon,et al.  Generative Adversarial Imitation Learning , 2016, NIPS.

[74]  Wolfram Burgard,et al.  Socially compliant mobile robot navigation via inverse reinforcement learning , 2016, Int. J. Robotics Res..

[75]  Vladlen Koltun,et al.  Playing for Data: Ground Truth from Computer Games , 2016, ECCV.

[76]  Richard F. Hartl,et al.  A survey on dynamic and stochastic vehicle routing problems , 2016 .

[77]  Vidya N. Murali,et al.  DeepLanes: End-To-End Lane Position Estimation Using Deep Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[78]  Marco Pavone,et al.  Control of robotic mobility-on-demand systems: A queueing-theoretical perspective , 2014, Int. J. Robotics Res..

[79]  J. Christian Gerdes,et al.  Shared Steering Control Using Safe Envelopes for Obstacle Avoidance and Vehicle Stability , 2016, IEEE Transactions on Intelligent Transportation Systems.

[80]  Amnon Shashua,et al.  Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving , 2016, ArXiv.

[81]  Emilio Frazzoli,et al.  A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles , 2016, IEEE Transactions on Intelligent Vehicles.

[82]  Andreas Lawitzky,et al.  A Game-Theoretic Approach to Replanning-Aware Interactive Scene Prediction and Planning , 2016, IEEE Transactions on Vehicular Technology.

[83]  Daniel Axehill,et al.  Interaction aware trajectory planning for merge scenarios in congested traffic situations , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[84]  Antonio M. López,et al.  The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[85]  Luis Herranz,et al.  Scene Recognition with CNNs: Objects, Scales and Dataset Bias , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[86]  Kyunghyun Cho,et al.  Query-Efficient Imitation Learning for End-to-End Autonomous Driving , 2016, ArXiv.

[87]  Sanjit A. Seshia,et al.  Towards Verified Artificial Intelligence , 2016, ArXiv.

[88]  Nidhi Kalra,et al.  Driving to Safety , 2016 .

[89]  Marco Pavone,et al.  Routing autonomous vehicles in congested transportation networks: structural properties and coordination algorithms , 2016, Autonomous Robots.

[90]  Anca D. Dragan,et al.  Planning for Autonomous Cars that Leverage Effects on Human Actions , 2016, Robotics: Science and Systems.

[91]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[92]  Eugenio Culurciello,et al.  ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation , 2016, ArXiv.

[93]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[94]  Alois Knoll,et al.  Tactical cooperative planning for autonomous highway driving using Monte-Carlo Tree Search , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[95]  Hannes Sommer,et al.  Predicting actions to act predictably: Cooperative partial motion planning with maximum entropy models , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[96]  Susanne Ernst,et al.  Towards a Functional System Architecture for Automated Vehicles , 2017, ArXiv.

[97]  Manfred Morari,et al.  Optimization‐based autonomous racing of 1:43 scale RC cars , 2015, ArXiv.

[98]  Marco Pavone,et al.  Risk-sensitive Inverse Reinforcement Learning via Coherent Risk Models , 2017, Robotics: Science and Systems.

[99]  Christoph Stiller,et al.  Decision making for autonomous driving considering interaction and uncertain prediction of surrounding vehicles , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[100]  Calin Belta,et al.  Minimum-violation scLTL motion planning for mobility-on-demand , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[101]  Charles Richter,et al.  Safe Visual Navigation via Deep Learning and Novelty Detection , 2017, Robotics: Science and Systems.

[102]  Yang Gao,et al.  End-to-End Learning of Driving Models from Large-Scale Video Datasets , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[103]  Roberto Cipolla,et al.  Concrete Problems for Autonomous Vehicle Safety: Advantages of Bayesian Deep Learning , 2017, IJCAI.

[104]  Jonathan A. DeCastro,et al.  Compositional and Contract-Based Verification for Autonomous Driving on Road Networks , 2017, ISRR.

[105]  Francesco Borrelli,et al.  A machine learning approach for personalized autonomous lane change initiation and control , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[106]  Dushyant Rao,et al.  Large-scale cost function learning for path planning using deep inverse reinforcement learning , 2017, Int. J. Robotics Res..

[107]  Edwin Olson,et al.  Multipolicy decision-making for autonomous driving via changepoint-based behavior prediction: Theory and experiment , 2015, Autonomous Robots.

[108]  Matthew Johnson-Roberson,et al.  Driving in the Matrix: Can virtual worlds replace human-generated annotations for real world tasks? , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[109]  Cewu Lu,et al.  Virtual to Real Reinforcement Learning for Autonomous Driving , 2017, BMVC.

[110]  Javier Alonso-Mora,et al.  Trajectory optimization for autonomous overtaking with visibility maximization , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[111]  John M. Dolan,et al.  Intention estimation for ramp merging control in autonomous driving , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[112]  Philip H. S. Torr,et al.  DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[113]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[114]  Alois Knoll,et al.  Deep neural networks for Markovian interactive scene prediction in highway scenarios , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[115]  Michael Gassner,et al.  SVO: Semidirect Visual Odometry for Monocular and Multicamera Systems , 2017, IEEE Transactions on Robotics.

[116]  Johann Marius Zöllner,et al.  Learning how to drive in a real world simulation with deep Q-Networks , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[117]  Lawrence D. Jackel,et al.  Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car , 2017, ArXiv.

[118]  Roland Siegwart,et al.  From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[119]  Mykel J. Kochenderfer,et al.  Imitating driver behavior with generative adversarial networks , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[120]  Javier Alonso-Mora,et al.  Predictive routing for autonomous mobility-on-demand systems with ride-sharing , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[121]  Mykel J. Kochenderfer,et al.  Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks , 2017, CAV.

[122]  Matthias Althoff,et al.  Ensuring drivability of planned motions using formal methods , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[123]  Michael W. Levin,et al.  Congestion-aware system optimal route choice for shared autonomous vehicles , 2017 .

[124]  Klaus C. J. Dietmayer,et al.  Probabilistic long-term prediction for autonomous vehicles , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[125]  Lennart Svensson,et al.  LIDAR-based driving path generation using fully convolutional neural networks , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[126]  Anca D. Dragan,et al.  Active Preference-Based Learning of Reward Functions , 2017, Robotics: Science and Systems.

[127]  Emilio Frazzoli,et al.  On-demand high-capacity ride-sharing via dynamic trip-vehicle assignment , 2017, Proceedings of the National Academy of Sciences.

[128]  Javier Alonso-Mora,et al.  Parallel autonomy in automated vehicles: Safe motion generation with minimal intervention , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[129]  Ingmar Posner,et al.  Find your own way: Weakly-supervised segmentation of path proposals for urban autonomy , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[130]  Jonathan P. How,et al.  Socially aware motion planning with deep reinforcement learning , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[131]  Nan Li,et al.  Game Theoretic Modeling of Driver and Vehicle Interactions for Verification and Validation of Autonomous Vehicle Control Systems , 2016, IEEE Transactions on Control Systems Technology.

[132]  Xiaojuan Qi,et al.  ICNet for Real-Time Semantic Segmentation on High-Resolution Images , 2017, ECCV.

[133]  Javier Alonso-Mora,et al.  Joint Multi-Policy Behavior Estimation and Receding-Horizon Trajectory Planning for Automated Urban Driving , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[134]  Sandy H. Huang,et al.  Enabling robots to communicate their objectives , 2017, Autonomous Robots.

[135]  Andreas Geiger,et al.  Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art , 2017, Found. Trends Comput. Graph. Vis..