A Software Architecture for Autonomous Vehicles: Team LRM-B Entry in the First CARLA Autonomous Driving Challenge

The objective of the first CARLA autonomous driving challenge was to deploy autonomous driving systems to lead with complex traffic scenarios where all participants faced the same challenging traffic situations. According to the organizers, this competition emerges as a way to democratize and to accelerate the research and development of autonomous vehicles around the world using the CARLA simulator contributing to the development of the autonomous vehicle area. Therefore, this paper presents the architecture design for the navigation of an autonomous vehicle in a simulated urban environment that attempts to commit the least number of traffic infractions, which used as the baseline the original architecture of the platform for autonomous navigation CaRINA 2. Our agent traveled in simulated scenarios for several hours, demonstrating his capabilities, winning three out of the four tracks of the challenge, and being ranked second in the remaining track. Our architecture was made towards meeting the requirements of CARLA Autonomous Driving Challenge and has components for obstacle detection using 3D point clouds, traffic signs detection and classification which employs Convolutional Neural Networks (CNN) and depth information, risk assessment with collision detection using short-term motion prediction, decision-making with Markov Decision Process (MDP), and control using Model Predictive Control (MPC).

[1]  Rüdiger Dillmann,et al.  Probabilistic MDP-behavior planning for cars , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[2]  Jianyu Chen,et al.  Intention-aware Long Horizon Trajectory Prediction of Surrounding Vehicles using Dual LSTM Networks , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[3]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[4]  Xiaohui Li,et al.  Real-Time Trajectory Planning for Autonomous Urban Driving: Framework, Algorithms, and Verifications , 2016, IEEE/ASME Transactions on Mechatronics.

[5]  Jan Albert Mulder,et al.  Sensor Fault Detection and Isolation using Adaptive Extended Kalman Filter , 2012 .

[6]  Mohan M. Trivedi,et al.  Convolutional Social Pooling for Vehicle Trajectory Prediction , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  Thierry Fraichard,et al.  From Reeds and Shepp's to continuous-curvature paths , 1999, IEEE Transactions on Robotics.

[8]  Sebastian Thrun,et al.  Model based vehicle detection and tracking for autonomous urban driving , 2009, Auton. Robots.

[9]  WuLing Huang,et al.  Autonomous vehicles testing methods review , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[10]  Dizan Vasquez,et al.  A survey on motion prediction and risk assessment for intelligent vehicles , 2014, ROBOMECH Journal.

[11]  Klaus C. J. Dietmayer,et al.  Grid-based DBSCAN for clustering extended objects in radar data , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[12]  Myoungho Sunwoo,et al.  Development of Autonomous Car—Part II: A Case Study on the Implementation of an Autonomous Driving System Based on Distributed Architecture , 2015, IEEE Transactions on Industrial Electronics.

[13]  Christoph Stiller,et al.  Towards Cooperative Motion Planning for Automated Vehicles in Mixed Traffic , 2017, ArXiv.

[14]  Germán Ros,et al.  CARLA: An Open Urban Driving Simulator , 2017, CoRL.

[15]  Lei Chen,et al.  Cooperative Intersection Management: A Survey , 2016, IEEE Transactions on Intelligent Transportation Systems.

[16]  Alexey Dosovitskiy,et al.  End-to-End Driving Via Conditional Imitation Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Mohan M. Trivedi,et al.  How Would Surround Vehicles Move? A Unified Framework for Maneuver Classification and Motion Prediction , 2018, IEEE Transactions on Intelligent Vehicles.

[18]  Emilio Frazzoli,et al.  A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles , 2016, IEEE Transactions on Intelligent Vehicles.

[19]  Denis Fernando Wolf,et al.  End-to-End Deep Learning Applied in Autonomous Navigation using Multi-Cameras System with RGB and Depth Images , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[20]  Sherali Zeadally,et al.  Autonomous Cars: Research Results, Issues, and Future Challenges , 2019, IEEE Communications Surveys & Tutorials.

[21]  Matthias Mayr,et al.  Lanelet2: A high-definition map framework for the future of automated driving , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[22]  Fernando Santos Osório,et al.  Calibration and multi-sensor fusion for on-road obstacle detection , 2017, 2017 Latin American Robotics Symposium (LARS) and 2017 Brazilian Symposium on Robotics (SBR).

[23]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[24]  James U. Korein,et al.  Robotics , 2018, IBM Syst. J..

[25]  Qingquan Li,et al.  A Sensor-Fusion Drivable-Region and Lane-Detection System for Autonomous Vehicle Navigation in Challenging Road Scenarios , 2014, IEEE Transactions on Vehicular Technology.

[26]  Nidhi Kalra,et al.  Driving to Safety , 2016 .

[27]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[28]  Eric P. Xing,et al.  CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-driving , 2018, ECCV.

[29]  Magnus Olsson,et al.  Behavior Trees for decision-making in Autonomous Driving , 2016 .

[30]  Ragunathan Rajkumar,et al.  Towards a viable autonomous driving research platform , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[31]  Sanjiv Singh,et al.  The 2005 DARPA Grand Challenge: The Great Robot Race , 2007 .

[32]  Bijun Li,et al.  Object detection and tracking using multi-layer laser for autonomous urban driving , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[33]  Keshav Bimbraw,et al.  Autonomous cars: Past, present and future a review of the developments in the last century, the present scenario and the expected future of autonomous vehicle technology , 2015, 2015 12th International Conference on Informatics in Control, Automation and Robotics (ICINCO).

[34]  Philip Koopman,et al.  Challenges in Autonomous Vehicle Testing and Validation , 2016 .

[35]  Martin Lauer,et al.  A Literature Review on the Prediction of Pedestrian Behavior in Urban Scenarios , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[36]  Domitilla Del Vecchio,et al.  Automated Vehicle-to-Vehicle Collision Avoidance at Intersections , 2011 .

[37]  Byron Boots,et al.  Imitation learning for agile autonomous driving , 2019, Int. J. Robotics Res..

[38]  Shinpei Kato,et al.  An Open Approach to Autonomous Vehicles , 2015, IEEE Micro.

[39]  Jae-Hwan Kim,et al.  Threat prediction algorithm based on local path candidates and surrounding vehicle trajectory predictions for automated driving vehicles , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[40]  Martin Strobl,et al.  OpenDRIVE 2010 and Beyond -Status and future of the de facto standard for the description of road networks , 2010 .

[41]  John M. Dolan,et al.  A behavioral planning framework for autonomous driving , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[42]  Shinpei Kato,et al.  Autoware on Board: Enabling Autonomous Vehicles with Embedded Systems , 2018, 2018 ACM/IEEE 9th International Conference on Cyber-Physical Systems (ICCPS).

[43]  Denis Wolf,et al.  Scene Compliant Trajectory Forecast With Agent-Centric Spatio-Temporal Grids , 2019, IEEE Robotics and Automation Letters.

[44]  Tao Zhang,et al.  A Flexible Multi-Layer Map Model Designed for Lane-Level Route Planning in Autonomous Vehicles , 2019, Engineering.

[45]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Sanjiv Singh,et al.  The DARPA Urban Challenge: Autonomous Vehicles in City Traffic, George Air Force Base, Victorville, California, USA , 2009, The DARPA Urban Challenge.

[47]  Vincent Frémont,et al.  Exploiting fully convolutional neural networks for fast road detection , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[48]  Jorge Godoy,et al.  A decision-making architecture for automated driving without detailed prior maps , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[49]  M. Murugan,et al.  A Review on Software Architecture Styles with Layered Robotic Software Architecture , 2015, 2015 International Conference on Computing Communication Control and Automation.

[50]  Hong Zhu,et al.  Software design methodology - from pinciples to architectural styles , 2005 .

[51]  Simon Fong,et al.  DBSCAN: Past, present and future , 2014, The Fifth International Conference on the Applications of Digital Information and Web Technologies (ICADIWT 2014).

[52]  Shengbo Eben Li,et al.  Pedestrian Trajectory Prediction with Learning-based Approaches: A Comparative Study , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[53]  Samyeul Noh,et al.  Decision-Making Framework for Autonomous Driving at Road Intersections: Safeguarding Against Collision, Overly Conservative Behavior, and Violation Vehicles , 2019, IEEE Transactions on Industrial Electronics.

[54]  Andreas Geiger,et al.  Efficient Large-Scale Stereo Matching , 2010, ACCV.

[55]  Xiaonian Wang,et al.  Road-Segmentation-Based Curb Detection Method for Self-Driving via a 3D-LiDAR Sensor , 2018, IEEE Transactions on Intelligent Transportation Systems.

[56]  Christoph Stiller,et al.  Deep Semantic Lane Segmentation for Mapless Driving , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[57]  Fernando Santos Osório,et al.  CaRINA Intelligent Robotic Car: Architectural design and applications , 2014, J. Syst. Archit..

[58]  E. Valuations.  A REVIEW ON EVALUATION METRICS FOR DATA CLASSIFICATION EVALUATIONS , 2015 .

[59]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[60]  Fei-Yue Wang,et al.  AI and Intelligent Vehicles Future Challenge (IVFC) in China: From cognitive intelligence to parallel intelligence , 2017, 2017 ITU Kaleidoscope: Challenges for a Data-Driven Society (ITU K).

[61]  Hairi Zamzuri,et al.  A review on threat assessment, path planning and path tracking strategies for collision avoidance systems of autonomous vehicles , 2018 .

[62]  Henk Wymeersch,et al.  Traffic Coordination at Road Intersections: Autonomous Decision-Making Algorithms Using Model-Based Heuristics , 2017, IEEE Intelligent Transportation Systems Magazine.

[63]  Nan Li,et al.  Game Theoretic Modeling of Driver and Vehicle Interactions for Verification and Validation of Autonomous Vehicle Control Systems , 2016, IEEE Transactions on Control Systems Technology.

[64]  Shuang Wu,et al.  Creating Autonomous Vehicle Systems , 2017, Synthesis Lectures on Computer Science.

[65]  Daniel J. Fagnant,et al.  Preparing a Nation for Autonomous Vehicles: Opportunities, Barriers and Policy Recommendations , 2015 .

[66]  Mykel J. Kochenderfer,et al.  Pedestrian Collision Avoidance System for Scenarios with Occlusions , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[67]  Jingda Wu,et al.  Multi-modal Sensor Fusion-Based Deep Neural Network for End-to-end Autonomous Driving with Scene Understanding , 2020, ArXiv.

[68]  Denis Fernando Wolf,et al.  Road terrain detection: Avoiding common obstacle detection assumptions using sensor fusion , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[69]  Fernando Santos Osório,et al.  Longitudinal and lateral control for autonomous ground vehicles , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[70]  Xin Chen,et al.  High definition maps in urban context , 2018, SIGSPACIAL.

[71]  Xiangmo Zhao,et al.  A 3D LiDAR Data-Based Dedicated Road Boundary Detection Algorithm for Autonomous Vehicles , 2019, IEEE Access.

[72]  Joshué Pérez,et al.  Making transport safer: V2V-based automated emergency braking system , 2011 .

[73]  Erik Rosen,et al.  Autonomous Emergency Braking for Vulnerable Road Users , 2013 .

[74]  Marin Toromanoff,et al.  End-to-End Model-Free Reinforcement Learning for Urban Driving Using Implicit Affordances , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[75]  Denis Fernando Wolf,et al.  Feature Detection for Vehicle Localization in Urban Environments Using a Multilayer LIDAR , 2016, IEEE Transactions on Intelligent Transportation Systems.

[76]  Nikolaos Papanikolopoulos,et al.  Fast segmentation of 3D point clouds: A paradigm on LiDAR data for autonomous vehicle applications , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[77]  Michael R. M. Jenkin,et al.  Computational principles of mobile robotics , 2000 .

[78]  Clément Zinoune,et al.  Lane-level map-matching with integrity on high-definition maps , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[79]  Paul C. Jorgensen,et al.  Software Testing: A Craftsman's Approach , 1995 .

[80]  C. Krishna Mohan,et al.  Vehicle Trajectory Prediction at Intersections using Interaction based Generative Adversarial Networks , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[81]  Hanna Kurniawati,et al.  An Online POMDP Solver for Uncertainty Planning in Dynamic Environment , 2013, ISRR.

[82]  Kikuo Fujimura,et al.  Predicting unexpected maneuver while approaching intersection , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[83]  Gijs Dubbelman,et al.  A Survey on Path Prediction Techniques for Vulnerable Road Users: From Traditional to Deep-Learning Approaches , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[84]  Bo Wahlberg,et al.  Clothoid-based model predictive control for autonomous driving , 2015, 2015 European Control Conference (ECC).

[85]  Makoto Obayashi,et al.  Appropriate overtaking motion generating method using predictive control with suitable car dynamics , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[86]  Christos Katrakazas,et al.  Real-time motion planning methods for autonomous on-road driving: State-of-the-art and future research directions , 2015 .

[87]  William Lewis,et al.  Software Testing and Continuous Quality Improvement , 2000 .

[88]  L Aarts,et al.  European truck platooning challenge , 2016 .

[89]  Sascha Wirges,et al.  Making Bertha Cooperate–Team AnnieWAY’s Entry to the 2016 Grand Cooperative Driving Challenge , 2018, IEEE Transactions on Intelligent Transportation Systems.

[90]  Mehrdad Dianati,et al.  A Survey of the State-of-the-Art Localization Techniques and Their Potentials for Autonomous Vehicle Applications , 2018, IEEE Internet of Things Journal.

[91]  Jonas Firl,et al.  Online maneuver recognition and multimodal trajectory prediction for intersection assistance using non-parametric regression , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.