Driver's view and vehicle surround estimation using omnidirectional video stream

Our research is focused on the development of novel machine vision based telematic systems, which provide non-intrusive probing of the state of the driver and driving conditions. In this paper we present a system which allows simultaneous capture of the driver's head pose, driving view, and surroundings of the vehicle. The integrated machine vision system utilizes a video stream of full 360 degree panoramic field of view. The processing modules include perspective transformation, feature extraction, head detection, head pose estimation, driving view synthesis, and motion segmentation. The paper presents a multi-state statistical decision models with Kalman filtering based tracking for head pose detection and face orientation estimation. The basic feasibility and robustness of the approach is demonstrated with a series of systematic experimental studies.

[1]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[2]  W. Krüger.  Robust real-time ground plane motion compensation from a moving vehicle , 1999, Machine Vision and Applications.

[3]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[4]  Thiagalingam Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation , 2001 .

[5]  Mohan M. Trivedi,et al.  Video arrays for real-time tracking of person, head, and face in an intelligent room , 2003, Machine Vision and Applications.

[6]  R. A. McLaughlin,et al.  Randomized Hough transform: better ellipse detection , 1996, Proceedings of Digital Processing Applications (TENCON '96).

[7]  Gaudenz Danuser,et al.  Parametric Model Fitting: From Inlier Characterization to Outlier Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[9]  Mohan M. Trivedi,et al.  DRIVER HEAD POSE AND VIEW ESTIMATION WITH SINGLE OMNIDIRECTIONAL VIDEO STREAM , 2003 .

[10]  Alex Pentland,et al.  Coupled hidden Markov models for complex action recognition , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Mohan M. Trivedi,et al.  Streaming face recognition using multicamera video arrays , 2002, Object recognition supported by user interaction for service robots.

[12]  Mohan M. Trivedi,et al.  Real-time driver affect analysis and tele-viewing system , 2003, IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No.03TH8683).

[13]  Emanuele Trucco,et al.  Introductory techniques for 3-D computer vision , 1998 .

[14]  Marian Stewart Bartlett,et al.  Classifying Facial Actions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Mohan M. Trivedi,et al.  Intelligent environments and active camera networks , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[16]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..