Socially-Compatible Behavior Design of Autonomous Vehicles With Verification on Real Human Data

As more and more autonomous vehicles (AVs) are being deployed on public roads, designing socially compatible behaviors for them is becoming increasingly important. In order to generate safe and efficient actions, AVs need to not only predict the future behaviors of other traffic participants, but also be aware of the uncertainties associated with such behavior prediction. In this letter, we propose an uncertain-aware integrated prediction and planning (UAPP) framework. It allows the AVs to infer the characteristics of other road users online and generate behaviors optimizing not only their own rewards, but also their courtesy to others, and their confidence regarding the prediction uncertainties. We first propose the definitions for courtesy and confidence. Based on that, their influences on the behaviors of AVs in interactive driving scenarios are explored. Moreover, we evaluate the proposed algorithm on naturalistic human driving data by comparing the generated behavior against ground truth. Results show that the online inference can significantly improve the human-likeness of the generated behaviors. Furthermore, we find that human drivers show great courtesy to others, even for those without right-of-way. We also find that such driving preferences vary significantly in different cultures.

[1]  Wei Ji Ma,et al.  Confidence reports in decision-making with multiple alternatives violate the Bayesian confidence hypothesis , 2019, bioRxiv.

[2]  R. Duncan Luce,et al.  Individual Choice Behavior: A Theoretical Analysis , 1979 .

[3]  Masayoshi Tomizuka,et al.  Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning With Application to Autonomous Driving , 2020, IEEE Robotics and Automation Letters.

[4]  Anca D. Dragan,et al.  Information gathering actions over human internal state , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Anca D. Dragan,et al.  Courteous Autonomous Cars , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Tatsuo Arai,et al.  Human-Robot Collision Avoidance using a modified Social force Model with Body Pose and Face Orientation , 2013, Int. J. Humanoid Robotics.

[7]  R. Bhagat.  Culture's Consequences: Comparing Values, Behaviors, Institutions, and Organizations Across Nations , 2002 .

[8]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Wolfram Burgard,et al.  Socially compliant mobile robot navigation via inverse reinforcement learning , 2016, Int. J. Robotics Res..

[10]  Jonathan P. How,et al.  Socially aware motion planning with deep reinforcement learning , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  Jarrod M. Snider.  Automatic Steering Methods for Autonomous Automobile Path Tracking , 2009 .

[12]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[13]  Lijun Sun,et al.  On Social Interactions of Merging Behaviors at Highway On-Ramps in Congested Traffic , 2020, IEEE Transactions on Intelligent Transportation Systems.

[14]  Silvio Savarese,et al.  Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Anca D. Dragan,et al.  Planning for Autonomous Cars that Leverage Effects on Human Actions , 2016, Robotics: Science and Systems.

[16]  S. Roth,et al.  Driving Style Encoder: Situational Reward Adaptation for General-Purpose Planning in Automated Driving , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Masayoshi Tomizuka,et al.  Interaction-Aware Behavior Planning for Autonomous Vehicles Validated with Real Traffic Data , 2020, ArXiv.

[18]  R. Luce,et al.  Individual Choice Behavior: A Theoretical Analysis. , 1960 .

[19]  A. Tversky,et al.  Advances in prospect theory: Cumulative representation of uncertainty , 1992 .

[20]  Nando de Freitas,et al.  Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning , 2018, ICML.

[21]  Adam Kepecs,et al.  A computational framework for the study of confidence in humans and animals , 2012, Philosophical Transactions of the Royal Society B: Biological Sciences.

[22]  Sergey Levine,et al.  Continuous Inverse Optimal Control with Locally Optimal Examples , 2012, ICML.

[23]  M. Tomizuka,et al.  Adaptive sampling-based motion planning with a non-conservatively defensive strategy for autonomous driving , 2020 .

[24]  P. Molnár.  Social Force Model for Pedestrian Dynamics Typeset Using Revt E X 1 , 1995 .

[25]  Gonzalo Ferrer,et al.  Proactive kinodynamic planning using the Extended Social Force Model and human motion prediction in urban environments , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Jean Oh,et al.  Social Attention: Modeling Attention in Human Crowds , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[27]  Masayoshi Tomizuka,et al.  INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps , 2019, ArXiv.

[28]  Tianyu Gu,et al.  Improved Trajectory Planning for On-Road Self-Driving Vehicles Via Combined Graph Search, Optimization & Topology Analysis , 2017 .

[29]  Wei Zhan,et al.  A Fast Integrated Planning and Control Framework for Autonomous Driving via Imitation Learning , 2017, Volume 3: Modeling and Validation; Multi-Agent and Networked Systems; Path Planning and Motion Control; Tracking Control Systems; Unmanned Aerial Vehicles (UAVs) and Application; Unmanned Ground and Aerial Vehicles; Vibration in Mechanical Systems; Vibrat.

[30]  Wei Zhan,et al.  Behavior Planning of Autonomous Cars with Social Perception , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[31]  G. Hofstede.  Culture′s Consequences: Comparing Values, Behaviors, Institutions and Organizations Across Nations , 2001 .

[32]  Wei Zhan,et al.  Probabilistic Prediction of Interactive Driving Behavior via Hierarchical Inverse Reinforcement Learning , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[33]  Malte Risto,et al.  The social behavior of autonomous vehicles , 2016, UbiComp Adjunct.

[34]  Evgueni A. Haroutunian,et al.  Information Theory and Statistics , 2011, International Encyclopedia of Statistical Science.

[35]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[36]  Daniela Rus,et al.  Dynamic Risk Density for Autonomous Navigation in Cluttered Environments without Object Detection , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[37]  Wei Zhan,et al.  A non-conservatively defensive strategy for urban autonomous driving , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[38]  Masayoshi Tomizuka,et al.  Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safety , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).