GLADAS: Gesture Learning for Advanced Driver Assistance Systems

Human-computer interaction (HCI) is crucial for the safety of lives as autonomous vehicles (AVs) become commonplace. Yet, little effort has been put toward ensuring that AVs understand humans on the road. In this paper, we present GLADAS, a simulator-based research platform designed to teach AVs to understand pedestrian hand gestures. GLADAS supports the training, testing, and validation of deep learning-based self-driving car gesture recognition systems. We focus on gestures as they are a primordial (i.e, natural and common) way to interact with cars. To the best of our knowledge, GLADAS is the first system of its kind designed to provide an infrastructure for further research into human-AV interaction. We also develop a hand gesture recognition algorithm for self-driving cars, using GLADAS to evaluate its performance. Our results show that an AV understands human gestures 85.91% of the time, reinforcing the need for further research into human-AV interaction.

[1]  Matthew Johnson-Roberson,et al.  Driving in the Matrix: Can virtual worlds replace human-generated annotations for real world tasks? , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Ahmet Gunduz,et al.  Real-time Hand Gesture Detection and Classification Using Convolutional Neural Networks , 2019, 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019).

[3]  Stephan Winter,et al.  Conventionalized gestures for the interaction of people in traffic with autonomous vehicles , 2016, IWCTS@SIGSPATIAL.

[4]  Germán Ros,et al.  CARLA: An Open Urban Driving Simulator , 2017, CoRL.

[5]  Susan T Chrysler,et al.  Creating Pedestrian Crash Scenarios in a Driving Simulator Environment , 2015, Traffic injury prevention.

[6]  Sowmya Somanath,et al.  Communicating Awareness and Intent in Autonomous Vehicle-Pedestrian Interaction , 2018, CHI.

[7]  Ahmet M. Kondoz,et al.  Fusion of LiDAR and Camera Sensor Data for Environment Sensing in Driverless Vehicles , 2017, ArXiv.

[8]  Toon Goedemé,et al.  Exploring RGB+Depth Fusion for Real-Time Object Detection , 2019, Sensors.

[9]  Rania Hussein,et al.  Applications of Image Processing and Real-Time embedded Systems in Autonomous Cars: A Short Review , 2017 .

[10]  Morteza Lahijanian,et al.  Social Trust: A Major Challenge for the Future of Autonomous Systems , 2016, AAAI Fall Symposia.

[11]  John K. Tsotsos,et al.  Autonomous Vehicles That Interact With Pedestrians: A Survey of Theory and Practice , 2018, IEEE Transactions on Intelligent Transportation Systems.

[12]  Fan Guo,et al.  Gesture Recognition for Chinese Traffic Police , 2015, 2015 International Conference on Virtual Reality and Visualization (ICVRV).

[13]  Ashish Kapoor,et al.  AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles , 2017, FSR.

[14]  Girish Chowdhary,et al.  Intent Communication between Autonomous Vehicles and Pedestrians , 2017, ArXiv.

[15]  Dinesh Manocha,et al.  Predicting Pedestrian Trajectories Using Velocity-Space Reasoning , 2012, WAFR.

[16]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[17]  Patrick Heinemann,et al.  Context-based detection of pedestrian crossing intention for autonomous driving in urban environments , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[18]  John K. Tsotsos,et al.  Understanding Pedestrian Behavior in Complex Traffic Scenes , 2018, IEEE Transactions on Intelligent Vehicles.

[19]  Rikard Fredriksson,et al.  Communicating Intent of Automated Vehicles to Pedestrians , 2018, Front. Psychol..

[20]  John K. Tsotsos,et al.  Agreeing to cross: How drivers and pedestrians communicate , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[21]  Dinesh Manocha,et al.  BRVO: Predicting pedestrian trajectories using velocity-space reasoning , 2015, Int. J. Robotics Res..

[22]  Melissa Cefkin,et al.  Developing Socially Acceptable Autonomous Vehicles , 2016 .

[23]  Russ Tedrake,et al.  Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation , 2018, NeurIPS.

[24]  Pedro J. Navarro,et al.  A Systematic Review of Perception System and Simulators for Autonomous Vehicles Research , 2019, Sensors.