Brake Maneuver Prediction – An Inference Leveraging RNN Focus on Sensor Confidence

In recent years, driver behavior analysis has led to countless driver assistance systems. In these systems, earlier detection of a driver’s maneuver intentions offers opportunities to improve driving experience and safety. Especially brake maneuvers are of fundamental importance because they are directly related to the avoidance of potential hazards.Current state-of-the-art brake assistance systems rely on the release speed of accelerator pedal as an indicator whether a brake event is planned. However, this simple and practical algorithm, fails to capture the overall movement pattern of accelerator pedal behaviors and cannot utilize rich information from different vehicle sensors.To address this issue, we propose a novel recurrent neural network architecture for the purpose of brake maneuver prediction. The proposed method exploits the advantages of multiple sensors. Unlike conventional practices where all signals are aggregated to a single neural network, we leverage the confidence of each sensor. We evaluate our approach based on a dataset of 44 drivers, comprising around 500 hours of naturalistic driving data. The evaluation results show that the proposed algorithm outperforms baseline method by large margin.

[1]  Bernadette A. Thomas,et al.  Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010 , 2012, The Lancet.

[2]  Mohan M. Trivedi,et al.  Computer Vision and Image Understanding xxx (2011) xxx–xxx Contents lists available at SciVerse ScienceDirect Computer Vision and Image Understanding , 2022 .

[3]  Rok Sosic,et al.  Drive2Vec: Multiscale State-Space Embedding of Vehicular Sensor Data , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[4]  Stefan Griesche,et al.  Evaluation of three different interaction designs for an automatic steering intervention , 2014 .

[5]  Berthold Färber,et al.  Driver perception and reaction in collision avoidance: Implications for ADAS development and testing , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[6]  James P Bliss,et al.  Alarm mistrust in automobiles: how collision alarm reliability affects driving. , 2003, Applied ergonomics.

[7]  Anurag Sai Vempati,et al.  StreetMap - Mapping and Localization on Ground Planes using a Downward Facing Camera , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[8]  Thomas Brox,et al.  Uncertainty Estimates and Multi-hypotheses Networks for Optical Flow , 2018, ECCV.

[9]  Berthold Färber,et al.  Automatic Emergency Steering with Distracted Drivers: Effects of Intervention Design , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[10]  Dot Hs,et al.  Analyses of Rear-End Crashes and Near-Crashes in the 100-Car Naturalistic Driving Study to Support Rear-Signaling Countermeasure Development , 2007 .

[11]  Martin Baumann,et al.  Interaction design of automatic steering for collision avoidance: challenges and potentials of driver decoupling , 2015 .

[12]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[13]  L. D. Adams,et al.  Review of the literature on obstacle avoidance maneuvers: braking versus steering. Final report , 1994 .

[14]  Daniel V. McGehee,et al.  Collision Warning Timing, Driver Distraction, and Driver Response to Imminent Rear-End Collisions in a High-Fidelity Driving Simulator , 2002, Hum. Factors.

[15]  Elgar Fleisch,et al.  Towards Device-Agnostic Mobile Cough Detection with Convolutional Neural Networks , 2019, 2019 IEEE International Conference on Healthcare Informatics (ICHI).

[16]  Klaus C. J. Dietmayer,et al.  Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[17]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[18]  Hema Swetha Koppula,et al.  Brain4Cars: Car That Knows Before You Do via Sensory-Fusion Deep Learning Architecture , 2016, ArXiv.

[19]  Naonori Ueda,et al.  Generalization error of ensemble estimators , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[20]  Nathalie Japkowicz,et al.  Anomaly Detection in Automobile Control Network Data with Long Short-Term Memory Networks , 2016, 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[21]  Tarak Gandhi,et al.  Looking-In and Looking-Out of a Vehicle: Computer-Vision-Based Enhanced Vehicle Safety , 2007, IEEE Transactions on Intelligent Transportation Systems.

[22]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[23]  Ravishankar K. Iyer,et al.  Hands Off the Wheel in Autonomous Vehicles?: A Systems Perspective on over a Million Miles of Field Data , 2018, 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN).

[24]  Felix Wortmann,et al.  Driver Identification via Brake Pedal Signals — A Replication and Advancement of Existing Techniques , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[25]  Mohan M. Trivedi,et al.  Driver Behavior and Situation Aware Brake Assistance for Intelligent Vehicles , 2007, Proceedings of the IEEE.

[26]  Mohan M. Trivedi,et al.  Pedal error prediction by driver foot gesture analysis: A vision-based inquiry , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[27]  Klaus C. J. Dietmayer,et al.  Leveraging Heteroscedastic Aleatoric Uncertainties for Robust Real-Time LiDAR 3D Object Detection , 2018, 2019 IEEE Intelligent Vehicles Symposium (IV).