Learning Interactive Driving Policies via Data-driven Simulation

Data-driven simulators promise high dataefficiency for driving policy learning. When used for modelling interactions, this data-efficiency becomes a bottleneck: Small underlying datasets often lack interesting and challenging edge cases for learning interactive driving. We address this challenge by proposing a simulation method that uses in-painted ado vehicles for learning robust driving policies. Thus, our approach can be used to learn policies that involve multi-agent interactions and allows for training via state-of-the-art policy learning methods. We evaluate the approach for learning standard interaction scenarios in driving. In extensive experiments, our work demonstrates that the resulting policies can be directly transferred to a full-scale autonomous vehicle without making use of any traditional sim-to-real transfer techniques such as domain randomization.

[1]  Vladlen Koltun,et al.  Learning by Cheating , 2019, CoRL.

[2]  Guy Rosman,et al.  Variational Autoencoder for End-to-End Control of Autonomous Driving with Novelty Detection and Training De-biasing , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[3]  Davide Scaramuzza,et al.  Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learning , 2021, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Daniela Rus,et al.  Learning Robust Control Policies for End-to-End Autonomous Driving From Data-Driven Simulation , 2020, IEEE Robotics and Automation Letters.

[5]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[6]  Andreas Geiger,et al.  Augmented Reality Meets Computer Vision: Efficient Data Generation for Urban Driving Scenes , 2017, International Journal of Computer Vision.

[7]  Alexey Dosovitskiy,et al.  End-to-End Driving Via Conditional Imitation Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Eric P. Xing,et al.  CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-driving , 2018, ECCV.

[9]  Ersin Yumer,et al.  GeoSim: Photorealistic Image Simulation with Geometry-Aware Composition , 2021, ArXiv.

[10]  Malte Risto,et al.  The social behavior of autonomous vehicles , 2016, UbiComp Adjunct.

[11]  Vladlen Koltun,et al.  On Offline Evaluation of Vision-based Driving Models , 2018, ECCV.

[12]  Yang Gao,et al.  End-to-End Learning of Driving Models from Large-Scale Video Datasets , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Alex Kendall,et al.  Urban Driving with Conditional Imitation Learning , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Sergey Tulyakov,et al.  Playable Video Generation , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Alex Bewley,et al.  Learning to Drive from Simulation without Real World Labels , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[16]  Raquel Urtasun,et al.  LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  John Lygeros,et al.  A Noncooperative Game Approach to Autonomous Racing , 2017, IEEE Transactions on Control Systems Technology.

[18]  Mayank Bansal,et al.  ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst , 2018, Robotics: Science and Systems.

[19]  Ali Farhadi,et al.  RoboTHOR: An Open Simulation-to-Real Embodied AI Platform , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Jitendra Malik,et al.  Gibson Env: Real-World Perception for Embodied Agents , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Radu Grosu,et al.  Model-based versus Model-free Deep Reinforcement Learning for Autonomous Racing Cars , 2021, ArXiv.

[22]  Cewu Lu,et al.  Virtual to Real Reinforcement Learning for Autonomous Driving , 2017, BMVC.

[23]  W. Zhang,et al.  Autonomous Robot Navigation Based on Multi-Camera Perception , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[24]  Ashish Mehta,et al.  Learning End-to-end Autonomous Driving using Guided Auxiliary Supervision , 2018, ICVGIP.

[25]  Vladlen Koltun,et al.  Deep Drone Racing: From Simulation to Reality With Domain Randomization , 2019, IEEE Transactions on Robotics.

[26]  Daniela Rus,et al.  Stochastic Dynamic Games in Belief Space , 2021, IEEE Transactions on Robotics.

[27]  Konstantin Sofiiuk,et al.  Foreground-aware Semantic Representations for Image Harmonization , 2020, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[28]  Sertac Karaman,et al.  FlightGoggles: Photorealistic Sensor Simulation for Perception-driven Robotics using Photogrammetry and Virtual Reality , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[29]  Matthew A. Brown,et al.  Learning to Segment via Cut-and-Paste , 2018, ECCV.

[30]  D. Manocha,et al.  AADS: Augmented autonomous driving simulation using data-driven algorithms , 2019, Science Robotics.

[31]  Fabien Moutarde,et al.  End to End Vehicle Lateral Control Using a Single Fisheye Camera , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[32]  Edwin Olson,et al.  Multipolicy decision-making for autonomous driving via changepoint-based behavior prediction: Theory and experiment , 2017, Auton. Robots.

[33]  Vicenç Puig,et al.  Autonomous racing using Linear Parameter Varying-Model Predictive Control (LPV-MPC) , 2020 .

[34]  John Lygeros,et al.  Efficient implementation of Randomized MPC for miniature race cars , 2016, 2016 European Control Conference (ECC).

[35]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[36]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[37]  Radu Grosu,et al.  Neural circuit policies enabling auditable autonomy , 2020, Nature Machine Intelligence.

[38]  Marin Toromanoff,et al.  End-to-End Model-Free Reinforcement Learning for Urban Driving Using Implicit Affordances , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  James M. Rehg,et al.  Autonomous Racing with AutoRally Vehicles and Differential Games , 2017, ArXiv.

[40]  Yi Xiao,et al.  Multimodal End-to-End Autonomous Driving , 2019, IEEE Transactions on Intelligent Transportation Systems.

[41]  Wojciech Zaremba,et al.  Domain randomization for transferring deep neural networks from simulation to the real world , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[42]  Ashish Kapoor,et al.  AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles , 2017, FSR.

[43]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[44]  Liam Paull,et al.  Learning Steering Bounds for Parallel Autonomous Systems , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[45]  Andreas Geiger,et al.  Conditional Affordance Learning for Driving in Urban Environments , 2018, CoRL.

[46]  Guy Rosman,et al.  Variational End-to-End Navigation and Localization , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[47]  David Janz,et al.  Learning to Drive in a Day , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[48]  Johann Marius Zöllner,et al.  Calibrating Uncertainty Models for Steering Angle Estimation , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[49]  Germán Ros,et al.  CARLA: An Open Urban Driving Simulator , 2017, CoRL.

[50]  Yuval Tassa,et al.  dm_control: Software and Tasks for Continuous Control , 2020, Softw. Impacts.

[51]  Roland Siegwart,et al.  From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[52]  Jitendra Malik,et al.  Habitat: A Platform for Embodied AI Research , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[53]  Raquel Urtasun,et al.  V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction , 2020, ECCV.

[54]  John Lygeros,et al.  Real-Time Control for Autonomous Racing Based on Viability Theory , 2017, IEEE Transactions on Control Systems Technology.

[55]  S. Karaman,et al.  Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Space , 2021, CoRL.

[56]  Sergey Levine,et al.  Deep Imitative Models for Flexible Inference, Planning, and Control , 2018, ICLR.