Semiautonomous Vehicular Control Using Driver Modeling

Threat assessment during semiautonomous driving is used to determine when correcting a driver's input is required. Since current semiautonomous systems perform threat assessment by predicting a vehicle's future state while treating the driver's input as a disturbance, autonomous controller intervention is limited to a restricted regime. Improving vehicle safety demands threat assessment that occurs over longer prediction horizons wherein a driver cannot be treated as a malicious agent. In this paper, we describe a real-time semiautonomous system that utilizes empirical observations of a driver's pose to inform an autonomous controller that corrects a driver's input when possible in a safe manner. We measure the performance of our system using several metrics that evaluate the informativeness of the prediction and the utility of the intervention procedure. A multisubject driving experiment illustrates the usefulness, with respect to these metrics, of incorporating the driver's pose while designing a semiautonomous system.

[1]  Rüdiger Dillmann,et al.  A probabilistic model for estimating driver behaviors and vehicle trajectories in traffic environments , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[2]  Allen Y. Yang,et al.  A Convex Optimization Framework for Active Learning , 2013, 2013 IEEE International Conference on Computer Vision.

[3]  Yihong Gong,et al.  Driving Safety Monitoring Using Semisupervised Learning on Time Series Data , 2010, IEEE Transactions on Intelligent Transportation Systems.

[4]  Deli Zhao,et al.  Graph Degree Linkage: Agglomerative Clustering on a Directed Graph , 2012, ECCV.

[5]  Francesco Borrelli,et al.  Robust Predictive Control for semi-autonomous vehicles with an uncertain driver model , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[6]  Sebastian Thrun,et al.  Stanley: The robot that won the DARPA Grand Challenge , 2006, J. Field Robotics.

[7]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[8]  Christoph Hermes,et al.  Long-term vehicle motion prediction , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[9]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[10]  Lotta Jakobsson,et al.  Collision Warning with Auto Brake - A Real-Life Safety Perspective , 2007 .

[11]  D Basacik,et al.  Smartphone Use While Driving: A Simulator Study , 2012 .

[12]  Cherian Varghese,et al.  An Examination of Driver Distraction as Recorded in NHTSA Databases , 2009 .

[13]  Lars Petersson,et al.  Vision in and out of Vehicles , 2003, IEEE Intell. Syst..

[14]  Alberto Bemporad,et al.  Model predictive control based on linear programming - the explicit solution , 2002, IEEE Transactions on Automatic Control.

[15]  Pravin Varaiya,et al.  Smart cars on smart roads: problems of control , 1991, IEEE Trans. Autom. Control..

[16]  Mohan M. Trivedi,et al.  Towards a vision-based system exploring 3D driver posture dynamics for driver assistance: Issues and possibilities , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[17]  J. C. Stutts,et al.  Driver inattention, driver distraction and traffic crashes , 2003 .

[18]  Hadas Kress-Gazit,et al.  Courteous Cars , 2008, IEEE Robotics & Automation Magazine.

[19]  H. Eric Tseng,et al.  A tube-based robust nonlinear predictive control approach to semiautonomous ground vehicles , 2014 .

[20]  Mohan M. Trivedi,et al.  On-road prediction of driver's intent with multimodal sensory cues , 2011, IEEE Pervasive Computing.

[21]  William Whittaker,et al.  Autonomous driving in urban environments: Boss and the Urban Challenge , 2008, J. Field Robotics.

[22]  Francesco Borrelli,et al.  Constrained Optimal Control of Linear and Hybrid Systems , 2003, IEEE Transactions on Automatic Control.

[23]  Hai Yu,et al.  Driving course prediction for vehicle handling maneuvers , 2012, 2012 American Control Conference (ACC).

[24]  J. Christian Gerdes,et al.  Equilibrium Analysis of Drifting Vehicles for Control Design , 2009 .

[25]  John D. Lee,et al.  Driver Distraction : Theory, Effects, and Mitigation , 2008 .

[26]  Osama Masoud,et al.  Vision-based methods for driver monitoring , 2003, Proceedings of the 2003 IEEE International Conference on Intelligent Transportation Systems.

[27]  Ulrich Kressel,et al.  Probabilistic trajectory prediction with Gaussian mixture models , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[28]  Mohan M. Trivedi,et al.  Computer Vision and Image Understanding xxx (2011) xxx–xxx Contents lists available at SciVerse ScienceDirect Computer Vision and Image Understanding , 2022 .

[29]  Maria E. Jabon,et al.  Facial expression analysis for predicting unsafe driving behavior , 2011, IEEE Pervasive Computing.

[30]  Mohan M. Trivedi,et al.  Tactical driver behavior prediction and intent inference: A review , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[31]  Thomas A. Dingus,et al.  The Impact of Driver Inattention on Near-Crash/Crash Risk: An Analysis Using the 100-Car Naturalistic Driving Study Data , 2006 .

[32]  D Basacik,et al.  Scoping study of driver distraction , 2008 .

[33]  Miguel Ángel Sotelo,et al.  Real-time system for monitoring driver vigilance , 2004, Proceedings of the IEEE International Symposium on Industrial Electronics, 2005. ISIE 2005..

[34]  N. Roy,et al.  Mobile Agent Trajectory Prediction using Bayesian Nonparametric Reachability Trees , 2011 .

[35]  Alex Pentland,et al.  Graphical models for driver behavior recognition in a SmartCar , 2000, Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No.00TH8511).

[36]  Mohan M. Trivedi,et al.  In-vehicle hand activity recognition using integration of regions , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[37]  Noelia Hernández,et al.  Gaze Fixation System for the Evaluation of Driver Distractions Induced by IVIS , 2012, IEEE Transactions on Intelligent Transportation Systems.

[38]  Mohan M. Trivedi,et al.  Trajectory Learning for Activity Understanding: Unsupervised, Multilevel, and Long-Term Adaptive Approach , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Rajesh Rajamani,et al.  Vehicle dynamics and control , 2005 .

[40]  Ruzena Bajcsy,et al.  Safe semi-autonomous control with enhanced driver modeling , 2012, 2012 American Control Conference (ACC).

[41]  S. Peeta,et al.  Adaptability of a hybrid route choice model to incorporating driver behavior dynamics under information provision , 2004, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[42]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[43]  Tomohiro Yamamura,et al.  A Driver Behavior Recognition Method Based on a Driver Model Framework , 2000 .

[44]  Sterling J. Anderson,et al.  An optimal-control-based framework for trajectory planning, threat assessment, and semi-autonomous control of passenger vehicles in hazard avoidance scenarios , 2010 .

[45]  Ming Li,et al.  Trajectory generation based vehicle action tracking for intersection driving safe , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[46]  R. Jafari,et al.  Body sensor networks for driver distraction identification , 2008, 2008 IEEE International Conference on Vehicular Electronics and Safety.