Fusing bio-inspired vision data for simplified high level scene interpretation: Application to face motion analysis

This paper proposes to demonstrate the advantages of using certain properties of the human visual system in order to develop a set of fusion algorithms for automatic analysis and interpretation of global and local facial motions. The proposed fusion algorithms rely on information coming from human vision models such as human retina and primary visual cortex previously developed at Gipsa-lab. Starting from a set of low level bio-inspired modules (static and moving contour detector, motion event detector and spectrum analyser) which are very efficient for video data pre-processing, it is shown how to organize them together in order to achieve reliable face motion interpretation. In particular, algorithms for global head motion analysis such as head nods, for local eye motion analysis such as blinking, for local mouth motion analysis such as speech lip motion and yawning and for open/close mouth/eye state detection are proposed and their performances are assessed. Thanks to the use of human vision model pre-processing which decorrelates visual information in a reliable manner, fusion algorithms are simplified and remain robust against traditional video acquisition problems (light changes, object detection failure, etc.).

[1]  James L. Crowley,et al.  Coordination of perceptual processes for computer mediated communication , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[2]  Te-Hsiu Sun,et al.  Face recognition using 2D and disparity eigenface , 2007, Expert Syst. Appl..

[3]  D. Hubel,et al.  Sequence regularity and geometry of orientation columns in the monkey striate cortex , 1974, The Journal of comparative neurology.

[4]  Alice Caplier,et al.  Biological approach for head motion detection and analysis , 2005, 2005 13th European Signal Processing Conference.

[5]  Dmitry O. Gorodnichy,et al.  Towards Automatic Retrieval of Blink-Based Lexicon for Persons Suffered from Brain-Stem Injury using Video Cameras , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[6]  Joseph J. Atick,et al.  Towards a Theory of Early Visual Processing , 1990, Neural Computation.

[7]  P. Caffier,et al.  Experimental evaluation of eye-blink parameters as a drowsiness measure , 2003, European Journal of Applied Physiology.

[8]  Gwen Littlewort,et al.  Automatic Recognition of Facial Actions in Spontaneous Expressions , 2006, J. Multim..

[9]  Jeanny Hérault,et al.  From retinal circuits to motion processing: a neuromorphic approach to velocity estimation , 1997, ESANN.

[10]  Alice Caplier,et al.  Jumping snakes and parametric model for lip segmentation , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[11]  Z. Zivkovic,et al.  A stabilized adaptive appearance changes model for 3D head tracking , 2001, Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems.

[12]  J. Bullier.  Integrated model of visual processing , 2001, Brain Research Reviews.

[13]  H Barlow,et al.  Redundancy reduction revisited , 2001, Network.

[14]  Mubarak Shah,et al.  Determining driver visual attention with one camera , 2003, IEEE Trans. Intell. Transp. Syst..

[15]  Zoran Saric,et al.  Adaptive microphone array based on pause detection , 2004 .

[16]  Alice Caplier,et al.  Head nods analysis: interpretation of non verbal communication gestures , 2005, IEEE International Conference on Image Processing 2005.

[17]  Bogdan Kwolek.  Model Based Facial Pose Tracking Using a Particle Filter , 2006, Geometric Modeling and Imaging--New Trends (GMAI'06).

[18]  Alice Caplier,et al.  Using Human Visual System modeling for bio-inspired low level image processing , 2010, Comput. Vis. Image Underst..

[19]  Z. Hammal,et al.  Eyes and eyebrows parametric models for automatic segmentation , 2004, 6th IEEE Southwest Symposium on Image Analysis and Interpretation, 2004..

[20]  Bruce A. Draper,et al.  FaceL: Facile Face Labeling , 2009, ICVS.

[21]  Gérard Bailly,et al.  Statistical active model for mouth components segmentation , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[22]  Jing Xiao,et al.  Robust full‐motion recovery of head by dynamic templates and re‐registration techniques , 2003 .

[23]  Bertrand Rivet.  La bimodalité de la parole au secours de la séparation de sources , 2006 .

[24]  Antoine Picot,et al.  Comparison between EOG and high frame rate camera for drowsiness detection , 2009, 2009 Workshop on Applications of Computer Vision (WACV).

[25]  Patrice Delmas,et al.  Towards robust lip tracking , 2002, Object recognition supported by user interaction for service robots.

[26]  Erik Hjelmås,et al.  Face Detection: A Survey , 2001, Comput. Vis. Image Underst..

[27]  Jeanny Hérault,et al.  Modeling Visual Perception for Image Processing , 2007, IWANN.

[28]  Daniela Gorski Trevisan,et al.  Multimodal focus attention and stress detection and feedback in an augmented driver simulator , 2007, Personal and Ubiquitous Computing.

[29]  Gwen Littlewort,et al.  Dynamics of Facial Expression Extracted Automatically from Video , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[30]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[31]  Dimitrios Tzovaras,et al.  Multimodal signal processing and interaction for a driving simulator: Component-based architecture , 2008, Journal on Multimodal User Interfaces.

[32]  Yaakob Shahrul Nizam,et al.  A face recognition system using template matching and neural network classifier , 2005 .

[33]  Jeffrey F. Cohn,et al.  Robust Lip Tracking by Combining Shape, Color and Motion , 2007 .

[34]  Gwen Littlewort,et al.  Drowsy Driver Detection Through Facial Movement Analysis , 2007, ICCV-HCI.

[35]  Chu Jiang-wei,et al.  A monitoring method of driver fatigue behavior based on machine vision , 2003, IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No.03TH8683).

[36]  Alice Caplier,et al.  Hypovigilence analysis: open or closed eye or mouth? Blinking or yawning frequency? , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[37]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..