Hybrid Verification Technique for Decision-Making of Self-Driving Vehicles

The evolution of driving technology has recently progressed from active safety features and ADAS systems to fully sensor-guided autonomous driving. Bringing such a vehicle to market requires not only simulation and testing but formal verification to account for all possible traffic scenarios. A new verification approach, which combines the use of two well-known model checkers: model checker for multi-agent systems (MCMAS) and probabilistic model checker (PRISM), is presented for this purpose. The overall structure of our autonomous vehicle (AV) system consists of: (1) A perception system of sensors that feeds data into (2) a rational agent (RA) based on a belief–desire–intention (BDI) architecture, which uses a model of the environment and is connected to the RA for verification of decision-making, and (3) a feedback control systems for following a self-planned path. MCMAS is used to check the consistency and stability of the BDI agent logic during design-time. PRISM is used to provide the RA with the probability of success while it decides to take action during run-time operation. This allows the RA to select movements of the highest probability of success from several generated alternatives. This framework has been tested on a new AV software platform built using the robot operating system (ROS) and virtual reality (VR) Gazebo Simulator. It also includes a parking lot scenario to test the feasibility of this approach in a realistic environment. A practical implementation of the AV system was also carried out on the experimental testbed.

[1]  Anand S. Rao,et al.  An Abstract Architecture for Rational Agents , 1992, KR.

[2]  Jonathan M. Aitken,et al.  Towards artificial situation awareness by autonomous vehicles , 2017 .

[3]  Sandor M. Veres,et al.  Reducing complexity of autonomous control agents for verifiability , 2016, ArXiv.

[4]  Mohan Manubhai Trivedi,et al.  When Vehicles See Pedestrians With Phones: A Multicue Framework for Recognizing Phone-Based Activities of Pedestrians , 2018, IEEE Transactions on Intelligent Vehicles.

[5]  Martin Lauer,et al.  A Literature Review on the Prediction of Pedestrian Behavior in Urban Scenarios , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[6]  Andrew Howard,et al.  Design and use paradigms for Gazebo, an open-source multi-robot simulator , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[7]  Affan Shaukat,et al.  Agent-Based Autonomous Systems and Abstraction Engines: Theory Meets Practice , 2016, TAROS.

[8]  Christoph Stiller,et al.  Automated Driving in Uncertain Environments: Planning With Interaction and Uncertain Maneuver Prediction , 2018, IEEE Transactions on Intelligent Vehicles.

[9]  Ricardo Omar Chávez García,et al.  Multiple Sensor Fusion and Classification for Moving Object Detection and Tracking , 2016, IEEE Transactions on Intelligent Transportation Systems.

[10]  Sandor M. Veres,et al.  Verification of logical consistency in robotic reasoning , 2016, Robotics Auton. Syst..

[11]  Gethin Norman,et al.  Strategy Synthesis for Autonomous Agents Using PRISM , 2018, NFM.

[12]  Sandor M. Veres,et al.  Autonomous vehicle control systems — a review of decision making , 2011 .

[13]  Ching-Yao Chan.  Advancements, prospects, and impacts of automated driving systems , 2017 .

[14]  Saeid Nahavandi,et al.  Intent Prediction of Pedestrians via Motion Trajectories Using Stacked Recurrent Neural Networks , 2018, IEEE Transactions on Intelligent Vehicles.

[15]  Michael Fisher,et al.  Formal verification of autonomous vehicle platooning , 2016, Sci. Comput. Program..

[16]  William Whittaker,et al.  Autonomous driving in urban environments: Boss and the Urban Challenge , 2008 .

[17]  S. Shankar Sastry,et al.  Formal methods for semi-autonomous driving , 2015, 2015 52nd ACM/EDAC/IEEE Design Automation Conference (DAC).

[18]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[19]  Jonathan Sprinkle,et al.  The CAT Vehicle Testbed: A Simulator with Hardware in the Loop for Autonomous Vehicle Applications , 2018, SCAV@CPSWeek.

[20]  Matthew Derry,et al.  Challenges in Perception and Decision Making for Intelligent Automotive Vehicles: A Case Study , 2016, IEEE Transactions on Intelligent Vehicles.

[21]  Emilio Frazzoli,et al.  A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles , 2016, IEEE Transactions on Intelligent Vehicles.

[22]  Marta Z. Kwiatkowska,et al.  A Framework for Verification of Software with Time and Probabilities , 2010, FORMATS.

[23]  Sanjiv Singh,et al.  The 2005 DARPA Grand Challenge: The Great Robot Race , 2007 .

[24]  Luke Fletcher,et al.  A perception-driven autonomous urban vehicle , 2008 .

[25]  Lounis Adouane.  Autonomous Vehicle Navigation: From Behavioral to Hybrid Multi-Controller Architectures , 2016 .

[26]  Mianxiong Dong,et al.  Humanlike Driving: Empirical Decision-Making System for Autonomous Vehicles , 2018, IEEE Transactions on Vehicular Technology.

[27]  Vivek Agarwal,et al.  A Cost-Effective Ultrasonic Sensor-Based Driver-Assistance System for Congested Traffic Conditions , 2009, IEEE Transactions on Intelligent Transportation Systems.

[28]  Clare Dixon,et al.  Formal Specification and Verification of Autonomous Robotic Systems , 2018, ACM Comput. Surv..

[29]  Anca D. Dragan,et al.  Planning for Autonomous Cars that Leverage Effects on Human Actions , 2016, Robotics: Science and Systems.

[30]  Michael Fisher,et al.  A Rational Agent Controlling an Autonomous Vehicle: Implementation and Formal Verification , 2017, FVAV@iFM.

[31]  Hongyang Qu,et al.  Local abstraction refinement for probabilistic timed programs , 2014, Theor. Comput. Sci..

[32]  Ulrich Kressel,et al.  Probabilistic trajectory prediction with Gaussian mixture models , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[33]  Yoshiko Kojima,et al.  SpaFIND: An Effective and Low-Cost Feature Descriptor for Pedestrian Protection Systems in Economy Cars , 2017, IEEE Transactions on Intelligent Vehicles.

[34]  Dariu Gavrila,et al.  Context-Based Pedestrian Path Prediction , 2014, ECCV.

[35]  Anca D. Dragan,et al.  Planning for cars that coordinate with people: leveraging effects on human actions for planning and active information gathering over human internal state , 2018, Autonomous Robots.

[36]  Pietro Perona,et al.  Fast Feature Pyramids for Object Detection , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Roland Siegwart,et al.  Toward automated driving in cities using close-to-market sensors: An overview of the V-Charge Project , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[38]  John K. Tsotsos,et al.  Agreeing to cross: How drivers and pedestrians communicate , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[39]  Noboru Ohnishi,et al.  A computer vision based vehicle detection and counting system , 2016, 2016 8th International Conference on Knowledge and Smart Technology (KST).

[40]  Javier Alonso-Mora,et al.  Planning and Decision-Making for Autonomous Vehicles , 2018, Annu. Rev. Control. Robotics Auton. Syst..

[41]  Michael Wooldridge,et al.  Introduction to multiagent systems , 2001 .

[42]  Sebastian Thrun,et al.  Towards fully autonomous driving: Systems and algorithms , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[43]  Ragunathan Rajkumar,et al.  Tools and Methodologies for Autonomous Driving Systems , 2018, Proceedings of the IEEE.

[44]  Affan Shaukat,et al.  Autonomous Nuclear Waste Management , 2018, IEEE Intelligent Systems.

[45]  Santokh Singh,et al.  Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey , 2015 .

[46]  Myoungho Sunwoo,et al.  Probabilistic Occupancy Filter for Parking Slot Marker Detection in an Autonomous Parking System Using AVM , 2019, IEEE Transactions on Intelligent Transportation Systems.

[47]  Ji Zhang,et al.  LOAM: Lidar Odometry and Mapping in Real-time , 2014, Robotics: Science and Systems.

[48]  Benedikt Nordhoff,et al.  Dijkstra’s Algorithm , 2013 .

[49]  Germán Ros,et al.  CARLA: An Open Urban Driving Simulator , 2017, CoRL.

[50]  Muhammad Abdullah,et al.  Navigation in human-robot and robot-robot interaction using optimization methods , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[51]  Michael Fisher,et al.  Verifying autonomous systems , 2013, CACM.

[52]  Thomas A. Runkler,et al.  Optimizing logistic processes using a fuzzy decision making approach , 2003, IEEE Trans. Syst. Man Cybern. Part A.

[53]  Byung-Sung Kim,et al.  Parking space detection using ultrasonic sensor in parking assistance system , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[54]  John K. Tsotsos,et al.  Understanding Pedestrian Behavior in Complex Traffic Scenes , 2018, IEEE Transactions on Intelligent Vehicles.

[55]  Alois Knoll,et al.  Vehicle detection based on LiDAR and camera fusion , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[56]  Rüdiger Dillmann,et al.  Learning Driver Behavior Models from Traffic Observations for Decision Making and Planning , 2015, IEEE Intelligent Transportation Systems Magazine.

[57]  Sebastian Thrun,et al.  Apprenticeship learning for motion planning with application to parking lot navigation , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[58]  Anand S. Rao,et al.  Modeling Rational Agents within a BDI-Architecture , 1997, KR.

[59]  Xiaowei Huang,et al.  Reasoning about Cognitive Trust in Stochastic Multiagent Systems , 2017, AAAI.

[60]  Amin Kianpisheh,et al.  Smart Parking System (SPS) Architecture Using Ultrasonic Detector , 2012 .

[61]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[62]  Luke Fletcher,et al.  A perception‐driven autonomous urban vehicle , 2008, J. Field Robotics.

[63]  Michael Fisher,et al.  Practical verification of decision-making in agent-based autonomous systems , 2013, Automated Software Engineering.

[64]  Sandor M. Veres,et al.  Natural Language Programming of Complex Robotic BDI Agents , 2012, Journal of Intelligent & Robotic Systems.

[65]  Ronald R. Mourant,et al.  A framework for modeling human-like driving behaviors for autonomous vehicles in driving simulators , 2001, AGENTS '01.

[66]  Stefano Soatto,et al.  Intent-aware long-term prediction of pedestrian motion , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[67]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.