Driving Posture Recognition by Joint Application of Motion History Image and Pyramid Histogram of Oriented Gradients

In the field of intelligent transportation system (ITS), automatic interpretation of a driver’s behavior is an urgent and challenging topic. This paper studies vision-based driving posture recognition in the human action recognition framework. A driving action dataset was prepared by a side-mounted camera looking at a driver’s left profile. The driving actions, including operating the shift lever, talking on a cell phone, eating, and smoking, are first decomposed into a number of predefined action primitives, that is, interaction with shift lever, operating the shift lever, interaction with head, and interaction with dashboard. A global grid-based representation for the action primitives was emphasized, which first generate the silhouette shape from motion history image, followed by application of the pyramid histogram of oriented gradients (PHOG) for more discriminating characterization. The random forest (RF) classifier was then exploited to classify the action primitives together with comparisons to some other commonly applied classifiers such as NN, multiple layer perceptron, and support vector machine. Classification accuracy is over 94% for the RF classifier in holdout and cross-validation experiments on the four manually decomposed driving actions.

[1]  Gary R. Bradski,et al.  Motion segmentation and pose recognition with motion history gradients , 2002, Machine Vision and Applications.

[2]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[3]  M.M. Trivedi,et al.  Visual context capture and analysis for driver attention monitoring , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[4]  Gary R. Bradski,et al.  Motion segmentation and pose recognition with motion history gradients , 2000, Proceedings Fifth IEEE Workshop on Applications of Computer Vision.

[5]  N. Papanikolopoulos,et al.  Driver activity monitoring through supervised and unsupervised learning , 2005, Proceedings. 2005 IEEE Intelligent Transportation Systems, 2005..

[6]  T. Kato,et al.  Detection of driver's posture in the car by using far infrared camera , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[7]  A V Desai,et al.  Vigilance monitoring for operator safety: A simulation study on highway driving. , 2006, Journal of safety research.

[8]  Jitendra Malik,et al.  Recognizing action at a distance , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[9]  Md. Atiqur Rahman Ahad,et al.  Motion recognition approach to solve overwriting in complex actions , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[10]  K. Torkkola,et al.  Driver inattention detection through intelligent analysis of readily available sensors , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[11]  B. Carnahan,et al.  A drowsy driver detection system for heavy vehicles , 1998, 17th DASC. AIAA/IEEE/SAE. Digital Avionics Systems Conference. Proceedings (Cat. No.98CH36267).

[12]  Osama Masoud,et al.  Vision-based methods for driver monitoring , 2003, Proceedings of the 2003 IEEE International Conference on Intelligent Transportation Systems.

[13]  Liang-Tien Chia,et al.  A new motion histogram to index motion content in video segments , 2005, Pattern Recognit. Lett..

[14]  Andrew Zisserman,et al.  Representing shape with a spatial pyramid kernel , 2007, CIVR '07.

[15]  Kang Ryoung Park,et al.  Vision-based method for detecting driver drowsiness and distraction in driver monitoring system , 2011 .

[16]  Bailing Zhang,et al.  Erratum to: Recognition of driving postures by combined features and random subspace ensemble of multilayer perceptron classifiers , 2012, Neural Computing and Applications.

[17]  Ying Wang,et al.  Human Activity Recognition Based on R Transform , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Kikuo Fujimura,et al.  Real-time pose classification for driver monitoring , 2002, Proceedings. The IEEE 5th International Conference on Intelligent Transportation Systems.

[19]  Mohan M. Trivedi,et al.  Computer Vision and Image Understanding xxx (2011) xxx–xxx Contents lists available at SciVerse ScienceDirect Computer Vision and Image Understanding , 2022 .

[20]  Somayeh Danafar,et al.  Action Recognition for Surveillance Applications Using Optic Flow and SVM , 2007, ACCV.

[21]  Yorgos Goletsis,et al.  Towards Driver's State Recognition on Real Driving Conditions , 2011 .

[22]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[23]  Barbara Caputo,et al.  Local velocity-adapted motion events for spatio-temporal recognition , 2007, Comput. Vis. Image Underst..

[24]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[25]  Ivan Laptev,et al.  On Space-Time Interest Points , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[26]  Liang Wang,et al.  Informative Shape Representations for Human Action Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[27]  Nikolaos Papanikolopoulos,et al.  Classifiers for driver activity monitoring , 2007 .

[28]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[29]  Mubarak Shah,et al.  Human Action Recognition in Videos Using Kinematic Features and Multiple Instance Learning , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Xiao Fan,et al.  Yawning Detection for Monitoring Driver Fatigue , 2007, 2007 International Conference on Machine Learning and Cybernetics.

[31]  J. Flusser,et al.  Moments and Moment Invariants in Pattern Recognition , 2009 .

[32]  Paul Watta,et al.  Nonparametric Approaches for Estimating Driver Pose , 2007, IEEE Transactions on Vehicular Technology.

[33]  Osama Masoud,et al.  A method for human action recognition , 2003, Image Vis. Comput..

[34]  Mohan M. Trivedi,et al.  Driving style recognition using a smartphone as a sensor platform , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[35]  Hsuan-Sheng Chen,et al.  Human action recognition using star skeleton , 2006, VSSN '06.

[36]  Bailing Zhang,et al.  Recognition of driving postures by contourlet transform and random forests , 2012 .

[37]  Mohan M. Trivedi,et al.  On the Roles of Eye Gaze and Head Dynamics in Predicting Driver's Intent to Change Lanes , 2009, IEEE Transactions on Intelligent Transportation Systems.

[38]  Mohan M. Trivedi,et al.  Multi-spectral and multi-perspective video arrays for driver body tracking and activity analysis , 2007, Comput. Vis. Image Underst..

[39]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[40]  David G. Stork,et al.  Pattern Classification , 1973 .

[41]  Margaret M. Peden,et al.  World Report on Road Traffic Injury Prevention , 2004 .

[42]  Mohan Trivedi,et al.  Driver activity analysis for intelligent vehicles: issues and development framework , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[43]  Gustav Markkula,et al.  Driver cognitive distraction detection: Feature estimation and implementation , 2007 .

[44]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[45]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..